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KOLLBACK-LEIBLER INFORMATION FUNCTION AND THE SEQUENTIAL 
SELECTION OF EXPERIMENTS TO .DISCRIMINATE AMONG 
SEVERAL LINEAR "MODELS 

Abstract 

by 

Steven Michael Sidik 

Assume that a finite sqt of potential linear models relating 
several controlled variables to an observed variable is postulated 
and that exactly one. of these models is the true model. The prob- 
lem is to sequentially design most informative experiments so that 
the correct model can be determined with as little experimentation 
as possible. We assume that the. error variance of the process is 
known. In addition, we assume the statistician possesses prior 
information which, can be expressed as the prior probability that 
each of the proposed models is indeed the correct model and prior 
multivariate normal distributions on the parameters of each of the 
postulated model equations. After each stage of sampling, the 
prior distributions and. the observed data values are used to com- 
pute posterior probabilities of the models being the true one and 
posterior distributions on the parameters of the models. Then 
sampling is terminated if either a prespecified number of observa- 
tions has been taken or if any of the posterior probabilities of 
the models exceeds a prespecified minimum stopping probability. 
Upon termination of sampling, the model with the largest posterior 
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probability is chosen to be the correct model. If sampling is not 

to be terminated, the next experiment chosen is that one in the set 

* 

of allowable values of the controlled variables which maximizes the 
expected Kullback-Leibler information function based upon the cur- 
rent posterior probabilities and distributions. 

An analytical study of this procedure is too complex and dif- 
ficult to adequately achieve. Hence a number of Monte-Carlo simu- 
lation experiments were performed to obtain information about the 
performance of this adaptive design procedure. Two basic types of 
Monte-Carlo experiments were performed. In the first, one of the 
models was chosen to be used to generate the random observations 
using known fixed values for the parameters. Then a large number 
of observations were taken using the Kullback-Leibler information 
functions as a criterion to choose the sequence of experiments. It 
was found the posterior probability of the chosen model relatively 
rapidly approaches the value of 1.0 and then fluctuates near 1.0. 

The posterior mean of the parameters of the correct model also 
rapidly approaches the known fixed values used to generate the ob- 
servations. In the second type of experiment, one of the models was 
chosen to be used to generate the random observations. Then for 
various combinations of the maximum number of observations, stopping 
probability, prior distributions of the parameters, and error vari- 
ance of the process, a large number of repetitions of the sequential 
design procedure were executed. Then a probability of correct se- 
lection and average sample number were calculated based unon the 

iii 
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number of times the procedure chose the correct model and the number 
of observations taken until termination. In general, it was found 
that as long as the prior mean of the correct model is not too dis- 
tant from the true value with respect to the means of the other 
models the probability of correct selection is respectably high. 
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CHAPTER 1 - INTRODUCTION AND LITERATURE SURVEY 

The general linear model has become one of the most useful 
statistical tools available to the modern scientific experimenter. 
There have been many books and papers written about techniques for 
choosing the appropriate or "best" linear model to fit to a set of 
data already collected. In general, these have been methods of 
hypothesis testing to determine which of a set of specified terms 
in a model equation may be dropped from the model. Much work has 
also been done with regard to the problem of designing best or 
optimal experiments to estimate the parameters of specified model 
equations. „ 

In this dissertation we study a sequential adaptive experi- 
mental design procedure for a related problem. Assume that a finite 
set of potential linear models relating certain controlled variables 
to an observed variable is postulated and that exactly one of these 
models is correct. The problem is to sequentially design most in- 
formative experiments so that the correct model equation can be de- 
termined with as little experimentation as possible. We also assume 
that the error variance of the process is known. In addition, we 
assume that the statistician possesses prior information which can 
be expressed by the prior probability that each of the proposed 
models is indeed the correct model and prior multivariate normal 
distributions on the parameters of the various models. We then de- 
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rive an adaptive procedure for designing the successive experiments 
using the Kullback-Leibler information function to maximize the 
anticipated information for discriminating among the models. That 
is, after each stage of sampling, the prior distributions and the 
observed values are used to compute posterior probabilities of the 
postulated models being correct and posterior distributions on the 
parameters of the models. Then if sampling is not to be terminated, 
the next experiment chosen is that which maximizes the expected 
Kullback-Leibler information based on the current posterior proba- 
bilities and distributions. Sampling., is terminated whenever either 
a prespecified number of observations is finally taken or whenever 
any of the posterior probabilities of the models exceeds a prespeci- 
fied value* Upon termination of sampling, the model with the larg- 
est posterior probability is chosen to be the correct model. 

An analytical study of this procedure is too complex and dif- 
ficult to adequately achieve. Hence a number of Monte-Carlo simula- 
tion experiments were performed to obtain information about the per- 
formance of this adaptive design procedure. Two basic types of 
Monte-Carlo experiments were performed. In the first, one of the 
models was chosen to be used to generate the random observations 
using known fixed values for the parameters. Then a large number 
of observations were taken using the Kullback-Leibler information 
as a criterion to choose the sequence of experiments. It was found 
the posterior probability of the chosen model relatively rapidly 
approaches the value of 1.0 and then fluctuates near 1.0. The 
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posterior mean of the parameters of the correct model also rapidly 
approach the known fixed values used to generate the observations. 

In the second type of experiment, one of the models was chosen to 
be used to generate the random observations. 'Then for various “com- 
binations of the maximum number of observations, stopping proba- 
bility, prior distributions of the parameters, and error variance 
of the process, a large nbmber of repetitions of the sequential 
design procedure were executed. Then a probability of correct se- 
lection and average sample number were calculated based upon the 
number of times the procedure chose the correct model and the num- 
ber of observations taken until termination. In general, it was 
found that as long as the prior mean of the correct model is not 
too distant from the true value with respect to the means of the 
other models the probability of correct selection is ; respectably 
high. 

We now briefly indicate the general organization of the 
dissertation. In Chapter 2 the notation used is described and the 
structure of the linear models is derived. Chapter 3 then develops 
the distribution theory which will be basic to the remainder of the 
dissertation. In particular, the posterior probabilities of the 
models, the posterior distributions of the parameters, and the 
Markovian nature of the sampling process are developed,. Some large 
sample results are then derived for the situation where the sequence 
of experiments is specified in advance of experimenting. These re- 
sults do not thus formally apply to the adaptive design procedure. 


We find, however, that they do appear to be true to a surprising 
extent and provide some help in explaining and interpreting the 
Monte-Carlo results. 

In Chapter 4, the Kullback-Leibler information concept is 
introduced and the derivation of the anticipated information as 
a function of the current posterior probabilities of the models 
and the current posterior distributions of the parameters is pre- 
sented. This anticipated information is the criterion function 
used to define the most informative experiment. Its use is dis- 

O 

cussed both from the point of view of its relation to the expected 
decrease in entropy and the point of view that, it results in a 
very simple function measuring the amount by which the expected 
value of the observed variable under each model is separated. 

The sequential experiment selection, stopping, and model se- 
lection rules are presented in Chapter 5. 

In Chapter 6, the Monte-Carlo simulation experiments are de- 
scribed and the results presented and discussed. Chapter 7 presents 
an example of application. Several appendixes are also included. 

Of most importance is appendix A which presents the computer pro- 
gram used to perform the simulation experiments. 

We now turn to a discussion of works by earlier authors who 
have considered similar problems. 

Lindley (1956) was one of . the first to consider the general 
idea of applying information concepts to the problems of statistical 
inference. He modified the concept of entropy and developed a num- 
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ber of interesting general results on the amount of information in 
an experiment about the parameters of the distribution of a random 
variable. 

Stone (1959) was one of the first to consider information con- 
cepts as applied to designing and comparing regression experiments. 
He used a Bayesian framex-7ork, but the problem he considers is that 
of parameter estimation rather than that of model selection. 

Another early and more relevant paper is that of Chernoff 
(1959) who applied the Kullback-Leibler information function to the 
sequential design of experiments when the cost of experimenting is 
small. His results are valid for the case of two terminal decisions 
and a finite number of experiments and states of nature. These re- 

C 

suits have been generalized by Albert (1961) to an infinite number 
of states of nature and by Bessler (1960) to an infinite number of 
experiments and k terminal actions. Kiefer and Sacks (1963) have 
also provided some extensions. 

The statement of Chernoff f s problem and the problem considered 
here are not identical and we proceed by analogizing his results to 
the problem at hand e In the context of the current problem, he 
would proceed by first assuming that at each stage of sampling the 
model with the largest posterior probability is the correct one. 

Then if A denotes the space of allowable experiments, define the 
Kullback-Leibler (K-L) information about model j in experiment 


aeA when model i is true as 
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I(a,i, j) 



f . (y|a)dy 


where f (via) denotes the probability density of y under model i 
when experiment aeA is performed. Let i denote the model with 
the highest current probability of being the correct one. Then in 
analogy to Chernof f , we define the optimal experiment as a(i) where 
a(i) is defined by that experiment satisfying 

l[a(l) , i,j] » sup inf I(a,i,k) 
aeA k^i 

That is, Chernoff represents the problem as a game between nature 
and the statistician where the statistician maximizes over A and 
nature minimizes over the alternative models assuming i is the 
correct model. Chernoff also specifically derives a stopping rule 
which we do not discuss here. 

Hunter and Reiner (1965) considered a sequential design pro- 
cedure for discriminating between- two model equations. Their proce- 
dure chooses the experimental conditions which, based upon maximum 
likelihood . estimates of the parameters from the data already col- 
lected, separate the expected values of the observed variable under 
the two models by as much as possible. 

Box and Hill (1967) discussed the use of the Kullback-Leibler 
information function, deriving it from considerations involving the 

* c 

entropy function. They consider the use of the K— L information 
function to sequentially discriminate among several mechanistic 
(nonlinear) model equations. Besides the fact that they consider 
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nonlinear models, their approach is different in the sense that al- 
though they do assume prior probabilities on the proposed models, 
and compute posterior probabilities from the observations, they as- 
sume the parameters of the model equations are known constants. 

Meeter, Pirie, and Blot (1970) have done a number of computer 
simulations comparing the methods of Chernoff and of Box and Hill. 
They found that the Box-Hill procedure performed quite well on the 
examples in comparison to Chernoff' s procedure.; It is interesting 
to note that Chernoff seems to be the only one of these authors 
who defined an explicit rule for terminating sampling. Although 
Chernoff 's procedure is known to be asymptotically optimal, it is 
also known to require very large sample sizes. 



CHAPTER 2 


STRUCTURE OF THE LINEAR MODELS 

In the theory of the general linear statistical model, we are 
concerned with problems involving model equations relating k con- 
trolled variables (z.; i » I, . « . ,k) to an observed variable (y) . 

1 

The form of the model equation is required to be 


J_ 

y = X/ 3 i h i (z l 5 * * c * ,Z k ) + 


i=l 

The known functions h_^ are arbitrary except that they may not con- 
tain any unknown parameters. The equation is linear in the unknown 
parameters $ and e is assumed to be a -random variable with ex- 
pectation zero and known finite variance. We may write 

x, = h.(z.,, «, . . ,z, ) and henceforth express the models in terms of 
ill k 

the x_^ variables. If n observations are made upon y we let 

x.. denote the value of x. at which the j 1 ' observation is made. 
Ji i 

Thus for the n observations the model may conveniently be written 


as 


= MB + e 


( 2 - 1 ) 


where 


i _ 


“ (yi>y 2 » • * *>y n > 
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11 

X 12 

* ' 

■ • 

i— i 
• — t 

21 

X 22 



X 2I 


X nl X n2 


9 B 2 > • ® • »3j) 

£ — (£^j£ 2> • o • , £^) 

and the are uncorrelated. The matrix M is called the design 

matrix for the experiment consisting of the u observations. The 
problem of experimental design is that of choosing the x values 
in some "optimal" manner. 

In certain situations in practice the experimeriter can postu- 
late several possible models involving different functions of the 
z variables which correspond to several possible mechanistic or 
empirically based theories. These may lead to the various models 
containing different sets of x_^. There may be some overlapping of 
the x^ among the models or there may be none. 

There are then two problems requiring solution. The first is 
that of choosing experiment designs which will enable the experi- 
menter to decide which of the potential models is the correct one. 
Then, having chosen the model, the parameters must be estimated. 

The second problem has many solutions using a variety of standard 
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techniques. This dissertation concerns itself primarily with a 
method of designing experiments to provide information for choosing 
the appropriate model equation. . 

We assume there are L different competing model equations. 

These models may be combined into one large possible model equa- 
tion and then the L hypothetical models are equivalent to there 
being L hypotheses restricting certain sets of parameters of the 
large model to be a priori zero* For example, we might have two 
controlled variables x^ and x^. And suppose the model equations 

C 

postulated are: ° 

V y - 6 i (1)x i + £ ; 

V y - *2 + e 

(3) (3) 

H 3 ; y ^1 X 1 + ^2 x 2 + C 

where denotes the coefficient of controlled variable i in 

i 

model equation j. The distinction must be made because although 
(i) (k) 

3. and 3 * are coefficients of variable i. their distribu- 
i x 

tions need not be the same. This notation is clumsy, however, and 
if we implicitly accept the fact that the distributions of the 3p^ 
depend upon the model, we may more simply rewrite the models as 


H r 

y = + e 

H 2 ' 

y = .3 2 x 2 + e 

v y ■ 

e i x l + e 2 X 2 • 


We say that models 1 and 2 are n este d with in model 3- This is 


o 



n 


equivalent to writing one model as y = + ^2 X 2 + E “ ^ + G 

and hypothesizing 

H r 8 2 - 0 

H 2 : P, - 0 

H 3 : 3 x + 0,3 2 t 0 

In this sense it is seen that the terms model and hypothesis are 
in t exchangeable and will be used interchangeably in the remainder 

o 

of this dissertation. The ^notation we adopt is that claims 

y *= + e 

where a is the appropriate k x 1 vector of ft's from B which 

X- Xj 

appear in model 1 and is the appropriate matrix of x's. 

We now precisely state the three basic distributional assump- 
tions about . the parameters and random variables of the models : 

(1) The vector e follows a multivariate normal distribution 
with mean 0 and precision matrix T. T is assumed known. (The 
precision matrix is the inverse of the covariance matrix of the 
distribution.) Since T ; must be positive definite symmetric, we 
need only consider the special case where T = tI since linear 
transformation of the y reduces all other cases to this one. Note 
that we assume x is known. Thus e - N(0,xT). 

(2) For each i - 1 , . « . ,L the prior distribution of oc^ is 

% ' N( ^,0*\,0 ) ! 


where y n and ¥ _ are known. 

Xf j U X/ j u 

u 

(3) The prior probability that the £ model is the correct 
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model equation is assumed specified and denoted by q* We re 

L 

quire 5 


1=1 


6 _ “ 1.0. In order to* satisfy this requirement in a 

!t,0 


completely precise manner, we must make the models mutually exclu- 
sive. As described so far, this need not be true. However, this 
is a simple problem to get around for the following reason. Each 

of the H specifies that a is an element of a k dimensional 
iL jC 

subset of K-space which we denote as E . For any pair H. and 

Jo 3 e 

we have either (1) E^E^, (2) E^!D E , or (3) neither space 

contains the other and E_.0 E^ has measure zero with respect to 

H. and H. . For case 1 define 9 . as 0 , = Pria.eE. - E, } . But 
3 k 3 3 3 3 k 

E, has zero measure with respect to H. and hence the distribution 
k J 

function of a. restricted to E. - E, is identical to the distri- 
3 3 k 

bution function of over all of . Thus for any practical 

purpose, the fact that E,3 E, does not affect any probability 

3 k * 

computations. Similar arguments apply to cases two and three. 

We now describe the space A of allowable experiments in 
more detail. If the number of elements of X is K, then a choice 
of experiment aeA is composed of the number J of observations 
to take and J vectors from some subset of Euclidean K-space. The 
J vectors specify the values of the controlled variables At 

the experiment or stage of experimenting the particular 

choice from A is denoted a,. 

3 



CHAPTER 3 


PREREQUISITE DISTRIBUTION THEORY 

In the remainder of this dissertation, much use 'will be made 
of the distribution of the observed variable, the posterior proba- 
bilities of the models, and the posterior distributions of the pa- 
rameters of the model equations. The first part of this chapter 
develops these distributions. The second part derives the fact that 
the sampling procedure is Markovian in nature and provides a nota- 
tion for describing the state of the process.. The third section of 
this chapter discusses some results on the limiting behavior of the 
posterior distributions when the sequence of experiments is chosen 
in advance. The strong restrictions that must be made to accomplish 
these large sample results and the fact that they do not describe 
the adaptive process might lead one to believe that they are not 
worthwhile pursuing. 

We find in chapter 6, however, .that there is a close analogy 
between these results and the behavior of the adaptive procedure, 
and that these results help explain and interpret the Monte Carlo 
simulation results. 


3.1 Posterior and Marginal Distributions 


Let f (y. la . jcO denote the density function of the 

Z j+1 1 j+1 l s 

vector y under H when the parameter values are given by 

3 A Jt 

at stage j+1 of sampling. Let the probability density function 
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of a 0 after j stages of sampling be denoted F .(a). This is 
Z z >3 

a preposterior density since it serves as the posterior density of 

a, after j stages of sampling and the prior density of a be- 
Z 

s t * ■ 

fore the j 4- 1 stage of sampling occurs. 

—y ■ 

Lemma 3.1: Aft er j stages of sampling, ag follows a multivariate 

~y 

normal distribution with mean vector u„ . and precision matrix 

LU 

. That is, after j stages of sampling, 




where 


ami 


. - + M 0 «TM„ . 


r . + r m’ .tm . 

, 0 y — ^ Z , i Z , i 

i=l 


V3 " + Vj-A-j-d 


= V 1 - 


j 


XKa 

i=l 


+ ' l, Jt,0 y £,0| 


(3-1) 


(3-2) 


and v?here M , denotes the design matrix specified by a under 

JC) 1 w 1 

H r 

-4- 

Proof : By Bayes theorem, if y^ is the observed vector at stage j 


h A « } = 






■ (3_3: 

.The symbol oc means "proportional to" and is used in the context oJ 
DeGroots (1970, p. 160) usage. Thus 
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5 *,j (a) 


•Q/2 


where (dropping subscripts) 

■ V y \ — V - 1 * r ^', 

Q « (Ma - y) T(Ma - y) + (a - y) ¥ (a - y) 

Since T and V are positive definite symmetric we can write 
Q » S’ (V + M T TM)a - 2 (ijT 1 4^ + y’TM)a + (terms not involving a) 


= [a - (y + M * TM) ”" X (K ' Ty + yy)]’ * Of + M*TM) 

• [a - (Y + m'tM) ^(M f Ty + ¥y) ] + (terms not involving a) 

The terms not involving a may be factored out through use of the 
proportionality device, leaving the kernel of a multivariate normal 
distribution with parameters as specified by the lemma. Thus 
is distributed as claimed. 

Q.E.D. 


Owen (1970) has derived a result similar to Lemma 3.1 in the 
case of a two factor experiment where the factors are treatments 
and blocks. 

We now turn to determining the distribution of y 
is done in two stages. First we do not know which of the models is 
in fact the correct one. Then for any given model, we do not know 
the value of a^. Let ^£^j+l I a j+l denote the distribution of 

-V ’ 

yjj-i under E Q when experiment a J _ L1 eA is performed and a fi is 


j+1 


j+1 


specified. Since we do not know we must average this distribu- 

i 

tion over all a„. Let f « (y . . . a , , , ) denote the mixture of the 
Z Z ^j+1 1 j+1 

densities ^£^yj+l i a j+l’^ with respect to the marginal posterior 

°f v 
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Lemma 3.2 The conditional distribution of y , given' and 

is a multivariate normal distribution with tnegi fr vector S^,. and 

precision matrix R . where 



Ki = 

Proof : The required mixture distribution is gjiven by 


[ + f '''"xl . T 

(3-4) 

+ h.j-d'S.-j-Aj-i 

(3-5) 


vW = S v ? j |8 j a ^.3-i 


- V 

(o)da 


S 


where 


-Q/2 ,•+ 

e da 


Q - “ M *,3 S) + ( “ ' Vi-d' -*>3-i <a ' Vj-d 

= + 40a - 2a' (M'TM + ¥) (m'tM + 4') _1 (vl’Ty + Ty) 


-v 4- 


+ y'Ty + y'Ty 


= a 1 (M’TM + :T)a - 2a ’ (M*TM + T) (m'tM + ¥) Vtf'Ty + Ty) 

-1„,» 






+ (M Ty + Ty) (M XM + ¥) (M Ty + Ty) 

- (M*Ty + ^’(m'tM + Y)” 1 (M , Ty + Ty) + y'Ty + y'Ty 

The first three terms yield the quadratic forrtf 

Q = [a - (M’TM + ¥) (M*Ty + Ty ) ] ’ * (m’tM 4- ?) 

• [a “ (M*TM + Ty + Ty)] 

-Ql/2 


The remainder of Q does not involve a and e 
kernel of a multivariate normal distribution h o that when e 


is the 

-V 2 
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is integrated over a we remain with 




where 


Q 2 = - (M 1 Ty + Vy) ' (m’tM + 'F)“ 1 (M , Ty + Vy) + y’Ty + jWy 


« y’[T - TM(M'TM + v F) _1 M , T]y - 2yTM(M 'TM H- y) 1 


-1 -* 

Ty 


+ (terms not involving y) 

= (y - s) R(y - s) + (terms not involving y) 0 

The terms not involving y may be factored out via the propor- 


tionality device leaving 


f (y.|a.) 
a j 1 j 




This is the kernel of a multivariate normal distribution with 

* —V 

mean vector $ and precision matrix R as claimed. Thus 

the density of y, given H and a. is given by 
J * J 

; < 2 '^ /2 iVji 1/2 “p{i^ -‘J} 


Q.E.D. 

Since the true model is unknown we now compute the mixture of 
the distributions of Lemma 3.2 with respect to the probabilities 


Lj 

■ D y 


(3-7) 


To compute the posterior probability of each model being 


cor- 


rect after the observation Yj +1 is obtained, we apply Bayes 
directly to get 




>e, 


e 


*,j+i 


r-'i+i 1 i+i 


Y f k (y j+ii a j+i ) 9 k,j 

k=l 


theorem 


(3-8) 


• 3»2 Markovian Nature of Sampling- Process 
Consider a sequence of random variables ^*^2* . • which 

take on values in a sample space or state space U. We let ^ de- 
note the o-field of subsets of S2 for which probabilities are de- 

O 

fined. The sequence of random variables V form a Markov Process 

if for every Fe5"" and for all . . . ,w^ in Q, and all for n, 

n = 1,2,3, . . . we have 

P.r{W ,, eFlw, = w- , . . . ,W = w ) 
n+1 1 1 1 * n n 

= Pr{W eF I W = w > 

n+1 1 « 


n n 


W” w n )dw 


(3-9) 


where g _(w|w ) denotes the generalized conditional probability 
n+1 n 

density function of W If the conditional probabilities in 

equation (3-9), equivalently the g , do not depend upon n the 
transition process is called stationary. The state space in this 
paper can be described by a vector containing: (1) the probabili- 

ties 0 , (2) the elements of the vectors describing the current 
posterior means under the various , and (3) the lower triangular 

n F fh p rnrrp^f n.-'.qfpri nr n ro on c i on msf ri cpc nnrlc r thp v p r i <-i i ! c: 



Thus 
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(1) ..(2) 


ft — I 1 * * * * ’^1 * * * * , ^*2 


( 1 ) ..( 2 ) 


V. 


,V 


(X) 


( k l) (1,1) 

r * * 1 




(k l ,k l ) 


i=l 


(i) 

6 . - 1.0, - ro < y^ J < + w > 

1 K- 


y . positive definite for i = 1, . . . ,L 


(3-10) 


For any given state wcfi the transition to the next state depends 
only upon the state w and the experiment a eA that is chosen. 

This is true because a determines the posterior precision 
matrices regardless of the value of y, and the posterior means 
p (y) and probabilities ?(y) are determined by equations (3-1) 
and (3 - 8) which again depend only upon w, y, and a. Thus the 
transition process on the states is Markovian. The process is 
stationary*, also > since for given initial g l the successive 
g 4t> Tlot depend upon n. The transition function may be described 
as follows. Define a mapping T:ft x Y ->■ ft and let Q(ft x Y) denote 
the Borel sets on ft x Y and Q(ft) denote the Borel sets on ft* Let 
T ^(F) denote the inverse image of F where FcQ(ft) and 
T _1 (F)eQ(ftxY) . Then if w' denotes the state of the system after 
sampling , 

Pr (w 1 eF | w ,a) = / 8^f ^(y ja,w)dy 

vj i=l 

(w,y) c ( F) 

3,3 Large Sample and Limiting Results 
Even though this paper is concerned primarily with small 
sample procedures, it is interesting and informative to know the 
large sample or limiting behavior of the parameters and me sampling 
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process. Unfortunately, for the adaptive procedure this is an ex- 
tremely difficult subject to study. Thus we do not study the adap- 
tive procedure here but instead consider the experiment selection 
procedure under the restrictions listed below in the hope that these 
results will illuminate the adaptive procedure in some sense. 

(1) Assume A is finite with N(A) elements, and represented 


as 


A = (a (1) ,a® 




(2) An infinite sequence {aj } is specified such that as the 

number of experiments approaches infinity, the proportion of times 
( i) 

that a is performed approaches p_^ with 0 < p^ < 1 and 

2p , =1.0. The experiments a. are chosen independently of each 
x - j 

© 

other. 


(3) Assume is the true model and that y is the true 

value of the parameters in the model. 

(4) Assume that only one observation is taken in each experi- 

_ (i) 

ment a . 


(5) Assume that the structure of A is such that all matrices 
under consideration are nonsingular. 

It should be noted that the most restrictive of the above as- 
sumptions is the second. For in a true sequential decision proce- 
dure, the actual experiment chosen is a random variable depending 
upon the previous observations obtained. Since we are in fact study- 
ing a problem other than the one of most importance the remainder of 

n ^ ^ ^ f- a i 1 T t 1 . . . . 1 . i ! ^ . 

“ " r ~ w ~ ^ _Ln A x. ^ i o o. Ut uciJ. i_ auu Uic I U.L Lcd 


o 




■nj-rn r^y , i . -t j 
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obtained cannot be rigorously applied to the sequential procedure. 
It will be seen in Chapter 6, however, that fairly extensive Monte 
Carlo simulations seem to bear up the general conclusions reached 
here. 


Let k(j) denote the superscript of the experiment performed 
at stage j. Thus if a Q = a^ 5 \ then k(10) = 5. Also let 
n(i,j) denote the number of times J is performed in the se- 


quence of experiments up to and including the j th stage. Let M 
denote the design matrix under H when a^ is chosen. 


l,i 


Lemma 3.3 Under the above assumptions the posterior precision 
matrices and mean vectors converge with probability one as j » 
to : 

o 

.N.(A) 


i-T + f „ 

jT &,j >■ r / P i £,i l,t 



v ! -i 1 p.M* .M.* .]p* 

\ / / i £,i i ,iJ r 

V-i / 

Proof: To prove the first limit, recall from equation (3-1) that 


& i J 


y + 

£,0 


TM «.,k(i) M ji > k(i) 


Thus 
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J 

2 — w ~ 2 — u / + \ ~ M * M 

jt £,j jx T £,0 [_j j AA £,k(i) 1 £,k(i) 

i=l 

N(A) 

= — f + V ^ M f M 

jx H,0 l_J 3 £.1 £,i 

i=l 

As j -*- «*, the first term goes to zero and the factors 
by assumption* 

To prove the second part note that y . - M,* .. , . * y + e. 

J 1 j K- \J / J 

x^here e_. ~ N(0,t)« Using the second form of equation (3-2) we get 


, = O', 


£,j 


o rl v XX k ( D y i 


+ \, 0 P £,0 


(3-11) 


Then substituting the expression for y^ into equation (3-11) gives 
(n(A) 


W i,j “ ( ' r £ 


a )_1 sD n(1 ’ J) ™U M i*,i 


V 


i=l 


+ T 


N(A) , j \ 

ly^ s « w j 

/ / £ , i\ / / m k(m),i, 

i=l Vn=l / 


4- ¥ u 

£,0 V £,0, 


vrtiere 6. . denotes the Kronecker delta function* Thus 


V, 


1 N(A) 

. - /“ ■ V . ) nlliH M * >M ; 

>3 \Ji £,3/ j/ / 3 £>* t j 1 


v 


N(A) / j \ 

>) / M £,i(j / / C m^k(m) , iy jx ^£,0^£,0 ^ 
i=l \ m=l / 


From assumption 2 we know that n(i,j) OT as j «> and since the 


"J. ■ ■ J ^py n ;* «* 
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form a sequence of independent and identically distributed ran- 
dom variables, the strong law of large numbers may be applied to 
show for i - 1, . . . ,L 

3 




e , = 0> = 1.0 

m k(m; ,i 


Since ~r— -* 0 as i ->■ «> °we then have 
JT 


M A) 

,-1/ m’ 

j 

V =1 


v 0 . 5! M. .M,* Ip* 

& >3 ^ \ ^ > i i » 


This sequence will not have a limit unless lim = p. exists 

4 3 


If such a limit exists, the lemma follows immediately. 


Q.E.D. 


Lemma 3.4 Under the assumptions stated, R : -*• t irrespective of 
- LsJ ' - 

{ a , } and 
3 


/N(A) 

S £,j ' M «,,k(j) ( V 




for large enough j . 

Proof : From equation (3-4) and the assumptions 


R ».i- Tl -°-. tM t.k(J) [ V3 +T Vk(J) M » 


r V 1 

>h( j) £,k(j 


As j -*■ ro , ^ k (j) M £ ]c(j)^ * and hence its inverse 

-> (0). But then R . *> x as claimed. 

» j 

From equation (3-5) 




f j j : — * - * i ■ 7^ .u - y * 7^ 
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For large enough j , K,k(j) M *,k<j> T + is asymptotl- 

cally like the identity matrix, I, so that 


ns (a) 




-i 


yy p.m' .m.* . 

/ / 1 £, i i ,x 


U-1 


We note that if £ - i , then from the definition of ¥ , we have 


s.* . - M,* , y.vU as expected, 

i »J i ,h(3> 


Q.EcD. 


Lemma 3.5 If H.* is the true hypothesis and the model of H.* 

l i_ 

is nested within the model of H , then under the above assumptions 


and assuming the parameter vectors are rearranged appropriately 

y 


Prrp.. , -M 

‘•j 


= 1.0 


Proof : From Lemma 3.3 we have 

C M (A) 

If the parameters are rearranged appropriately then M . may be 

X 5 1 


written 


M . » (M.* . ,M .) 
£,i i ,i £,i , 


where M . denotes the design matrix corresponding to the inde- 

X j 1 

pendent variables in H A but not in H.*. Thus 

£ l 


M. .M. . - 
. £ , i £ , i 


V,iV,i 


M . * .M, . 

l ,i £, l 

m’ . M . 

£ ; 1 £ , 1 
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and 


h ■ 


where 


r 

v 


y * 

i ,£ 


N(A) 


. ¥.* n 
l ,£ 


V* = Y; p . m ! *■ r M . * . 
i. *1 i ,x i ,1 


i=l 

N(A) 


Vv. - X) p i M I*,A,i 


and 


i=l 


N(A) 


% = D p A,A (i 

i-1 


Thus from a well known identity (e.g., Graybill (1969), p. 165) 

IK* - V.i^V.O' 1 J 


-1 


Also 


— •— ^ ^ ^ ^ ^ — — . — — W — —i 

:(\ - (\ - 


U (A) 


N(A) /p.M !* .M.* / 

d • d r.: 11 1 ,l 


i=l \p.M. .M.* . 
\ i £,x x ,i 



Thus 
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(v - v.aM*, J’\ 


-1 


- v, £ 

-(** - hvhk*,*) \*, t 

+ (\ - t )\*,l 

'“I 

_o, 

upon application of Lemma 3.6 which follows. Thus 

- (V P*\ 

* \</ \S j 

Q.E.D. 

L emma 3.6 (Problem 2.9 of Rao (1965)) If A and D are matrices 
possessing inverses, then 

(A + EDB ') - ” 1 = A _1 - A _ 1 B(B , a'“ 1 B + D~^) _ 1 B *A _1 
Proof : By direct multiplication we only need show 

I = (A + BDB’XA -1 - A _ 1 B(b'a“ 1 B -f D - 1 )" 1 B , A _1 

= i - b(b'a’" 1 b + d“ 1 )~ 1 b , a ~ 1 + bdb'a ” 1 

- bdb , a^ 1 b(b , a‘ 1 b + d” 1 )“ 1 b , a ” 1 
“ I - B[-(b'a -1 B + D -1 )" 1 + D - DB , A _ 1 B(B , A“ 1 B + d" 1 )“ 1 ]b'a " 1 


= I - B[D - [I + DB'A 1 B][B , A X B + D~ X ] 1 ]b'a 
= I ~ B[D - DfD^ 1 + b'a _ 1 B] [D’ 1 -f B , A“ 1 B3‘ 1 ]B , A" 1 


-1, 


L T, 1 . 1 


= I - B[D - D] B A 
= I 


-1 


Q.E.D. 
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To apply this result to Lemma 3.5 simply set ■- 


A = S'.* 
1 


B ■ V,, 

T) = _1 


We now turn to consideration of the limiting behavior of j ‘ 

Computer simulations for both nested and non-nested cases in- 
dicate that for any k where H^* is not nested in H^., 9^ r 0.0 

fairly rapidly and steadily. If Ih* is nested in then it 

c 

seems that 0 . ■+ 0.0. The rate is initially rapid but then be- 

k j J 

comes very slow and it behaves in a very erratic manner. These 
points are discussed in some detail in Chapter 6. 

It should be reiterated and these discussions have assumed 
the sequence {a^} to be specified and fixed for the sequence of ex- 
periments. In a sequential decision problem the sequence {a_„} is 
not fixed, but k(j) is in fact a random variable whose distribu- 
tion depends upon k(i) for i < j and the y^ for i < j <= 




CHAPTER 4 


ENTROPY FUNCTIONS AND THE KULLBACK-LEIBLER INFORMATION FUNCTION 

When comparing a number of experiments to determine which is 
the optimal one to perform, one must define optimal. In this dis- 
sertation, that experiment which yields the largest expected K-L 
information is defined as the optimal experiment. In particular, 
let I (w , a) denote the expected K-L information as a function of 
the experiment a and the current state w of the process. This 
function will be specified explicitly later. In this chapter, we 
first describe how the K-L information arises from attempting to 

o 

reduce the entropy of the probabilities of the models. We then 
develop an expression for I(w,a) and finally discuss the opera- 
tional meaning of the use of I(w,a) from a heuristic point of 
view. 

4.1 Development of the K-L Information Function 
The problem under consideration here is that we must choose 
one of a set of postulated model equations. For each model we have 
the posterior probability 0 , that it is the correct one. We 

would like to choose experiments which cause the posterior proba- 
bility of the correct model to increase most rapidly. An indirect 
method of accomplishing this is to choose experiments which most 
rapidly decrease the entropy of the set of probabilities 6^ . The 

entropy is defined as 
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c 
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£ (w) * - 


L 



In (8, .) 

^ J J 


It can be verified that the entropy attains a maximum when all the 

probabilities are equal and attains a minimum when any one of the 

probabilities is one and the rest are zero. 

Box and Hill (1967) proposed the use of the expected decrease 
between the entropy at the current stage of sampling and the antici- 
pated entropy at the next stage of sampling as the criterion for se- 
lection of experiments . They found, however, that the entropy func- 
tion is quite intractable analytically and applied a well-known in- 
equality to show the expected K-L information function provides an 
upper bound on the reduction of entropy. Let 6^(yjw,a) denote the 
posterior probability of model i if the value y is observed when 

-4. 

the state was w. Let w(y) denote the state of the process after 

observing the value y when it was in state w. Then the antici- 

pated entropy is given by 


E<j§[w(y) ,aj^ 



w,a) ]) f (y | w , a) dy 


Thus if the current state of the sampling process is w eft, and the 
experiment atA is performed, the expected decrease in entropy, 
R(w,a), is then defined as 


3a 


R(w,a) = <?(w) - E{<S[w(y) ,a] } 


^ ~ Xy 6i ln ^V + ^ J") e i (yk,a)ln[e i (y | w , 


a)] 


L 


a))dy 


k=l 


L, 

= " Z/ e i ln( V + 


1=1 


C' L 

r ,-i , i 

/ Xy G ^ ( ^ ,w,a)ln 

0 £ f,, (y |w.a) 

L 

/ 1=1 

B e k f k ( yi w - a) 

wk— 1 -J 


dy 



>dy 


(4-1) 


S fl iZ) e i f i (?|w * a)ln ^Li a ) ) J 

£=1 \i=l L 1 J 

by application of the following inequality (Kullback (1969), p. 15) 


Lt 

S i f £ (y|w,a) 1 n 


i=l 


ff £ (y[w,a)^ | 
f i (y |w,a) 


>. f p (y |w,a)ln 


f o (y|w,a) 


6 k f k^i w,a) 


k-1 


Let 



I(w,a,i,j) - / f' i (y|w,a)ln 


f i (y jw,a) 


f j Cy |^,a) 


dy 


(4-2) 


We note I(w,a,i,j) is defined as the expected amount of in- 
formation in the observations from experiment a for discriminating 
against in favor of Let ^ (w,a) denote the matrix whose 

j j fitment is I(w,a,i,j)« Then the inequality (4—1) may be 


written as 
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R(w,a) <_ 0^(w,a)e = I(w,a) (4-3) 

Meeter et al. (1970) proposed the fpllowing heuristic argument in 
favor of using I(w,a). If one knew that were indeed the cor- 

rect hypothesis and wished to maximize the information about 
for k ^ i, then it would be natural to maximize 


E 

k^i 


e k I(w } a,i,k) 


But since H, is assumed correct only with probability 0., it is 

1 o 3- 

equally natural to multiply the foregoing expression by 8^ and sum 
over i, But in doing this, one does end up with I(w,a). 

4.2 Evaluation of K-L Information Function 

■ 

From equation (3-6) we have (if y is Jxl.) that the density 
of y under . is .given by 

f n , ,, . -j/2 i ,i/2 -imy-S >'r j 

f^(y|a) » (2tt) |R £ | e 


Hence 


f /-^i \ -1/2 (y-s ) R (y-s ) 

f (y a) . . / m 7 m J m 

m 1 _ )d 1 1/2 1 .-1/2 e 

f n (^|a) * => n' -1/2 (y-s^) ’R^Cy-s^) 


n n 


Moreover 


In 


f <y|<0 

m 


f n (?|a) 


■ 2 (ln l R ml - H\l> 


- i<(y - s ) 'r (y - s ) 
2 J m m J m 


1 —V" — V I — ^ “V 

+ j (y - s ) R n (y - s ) 


(4-4) 
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and 


I(w,a,m,n) 



In 


R m ( y| a )l 


= E In 


L f n<*K 

f(yN 


f m (y|a)dy 




(4-5) 


where the expectation is taken under . the assumption y ~ N(s^,R m ). 

Note that I(w,a,m,m) = 0.0 for m - 1, . . . ,L. 

* ^ ")■ 

Lemma 4 . 1 If y ~ N(c.R) and R is positive definite, and Ac is 

a 

symmetric , then 

E{y'Ay} = tr (AR + c’Ac 
Proof By theorem 10.3.2 of Graybill (1969) 

E{(y - £) f A(y - c)} 



<y 


- ?)A(y - i )e -U2(y-b’R(y-t) d + 


But 


■ y f — v ■■ V - y y — V- 

E{ (y - c) A(y - c)} = E(y'Ay) - c 'Ac 


The lemma follows immediately. 

Q.E.D. 

Applying the lemma to the expectations of the quadratic forms 
in equation (4-4) we see: 

1. y ~ N(s ,R ) =$>E{(y - s ) *R ( y - s ) } - tr(R r’ 1 ) = J 
• nr m 7 m m J m mm 

2. y - £3 - N(s - "s ,R ) (y - s)R(y-s)} 

J n m n* m ^ w n n 

- tr (R R- 1 ) + (s - s' )'r (s - s ) 
n m m n n m n 
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Thus 


I(w,a,m,n) = ~ (ln\R I - ln|R \\ - \ J + ~ tr(R R 1 ) 
2 \ | m | Ini,/ 2 2 n m 


1 ^ | 

+ TT (s - s ) R (s ~ S ) 
2 m n n m n 


( 4 - 6 ) 


I (w,a >m,n) + I(w,a,n,m) « -J + “ tr (r R 1 ) + tr(R R™ 1 ) 

2 \ n m / V m n / 


+ i 


-v . 1 , -y -v- , -v -y „ « «y 

(s “ O R (s “ S ) + (S ~ S ) R (S 
m n nm n n m m n 


8 ) 
m 


- - j + y 


tr ( R R” 1 ) + tr (r r” 1 ) 

\ n m J \ m rl J 


+ -r k(s m - s ) (R + R ) (s 
2 m n m n m 


( 4 - 7 ) 


L n -1 


I(w,a) = 


X)Dv.'«- ,a,m } n) + I(w,a,n,m)] 


n=2 m=l 
L n-1 


■DDv.{-l[Kv;V«(v;‘) 

n=2 m=l V u 


+ \ [ (s - s ) ' (R + R ) (s 
2 m n m n m 


s ) ] 
n 


L n-1 



9 R |R 1 
m ml n 


-EDv.*iE*. 

n=2 m=l n=l 

L n-1 

+ T Y) Y) e 6 (s - s ) ’ (R + R )(s -t ) (4-8) 

2 / / j / n m m n m n m n v ' 

- n=2 m=l 

The last form of this equation appears to be the most convenient for 


computing purposes 
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4.3 Intuitive Analysis ° r ‘ 

Looking at the computing form of equation (4-8) it can be 

L n 

seen that there are three terms. The first term is -J 8 0 . 

n=2 m=T ' 

The value of this term does not depend upon a and hence has no 
effect upon the choice of a. From this consideration we note that 
computing the value of this term would not be beneficial if only 

c 

one more stage of experimentation is available* 

The third term of the sum is a weighted sum of the quadratic 


forms 


( s - s ) * (R + R ) (s - £3 ) 
m n m n m n 


Thus this term is in effect a separating function in the sense that 

these quadratic forms will be maximized when the pairs : of expected 

values of y under the various hypotheses are as far apart as pos- 

sible in comparison to the precisions of y. If the precisions 

and R are large then s and s do not need to be far apart to 

n ° m n r 

provide much information whereas if these precisions are small then 

the expected values s^ and s^ must be further apart to provide 

the same information. The weighting factors are the products 0 0 . 

r n m 

Thus when 0 and 0 are both small, 0 0 is very small and the 
n m n m J 

information due to the separation of s and s is discounted 

n m 


somewhat. 

If 

6 

n 

and 

6 

m 

separation 

of 

-V 

s 

n 

and 

->- 

s 

m 


third term causes experiments to a be chosen which separate the ex- 
pected values of y under the respective hypotheses which are still 
in serious contention for being chosen. 
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It is interesting to note that some authors (Hunt and Reiner 
(1965), e.g.) have proposed criteria for selection of experiments 
involving only distances between expected values. In a later paper, 
Box and Hill (1967) proposed that the distances as such are not im- 
portant, but the distances weighted by some function of the varia- 
bility about the expected values are important. It is seen here 
that the expected K-L inf ormation function does just that. 

The second term in equation (4-8) is 



This can be thought of as a weighted 


sum of ratios of precisions. If only one y value is to be ob- 
served, this component becomes 


1 

2 



(4-9) 


It would be interesting to see when this term is maximized. Upon 
taking partial derivatives of equation (4-9), setting to zero, and 
simplifying, one arrives at the following set of simultaneous non- 

O 

linear equations. 



k=l 


It can be immediately seen that one solution to this system is 


R-j- = R 2 - • * * = R^* This solution implies that the experiments 
should tend to give the same precision for the expected value of 
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under each hypothesis. This term is not considered any further 
here. 

In summary, it can be seen that the expected K-L information 
function in this case is basically a rather simple separating func- 
tion. One would be hard pressed to construct a much simpler sepa- 
rating function which has more intuitive appeal. If multivariate 
observations are permitted, then it might be possible to delete the 
second term of equation (4-8) to save a good deal of computings 




CHAPTER 5 


THE SEQUENTIAL DECISION PROCEDURE 
Three components are required for a sequential adaptive de- 
cision procedure; (1) a rule which determines if sampling should be 

o 

terminated or continued, (2) a rule which specifies the experiment 
to be performed given the current state of the system, and (3) a 
Q rule which selects the model equation which will be claimed to be 
true when sampling is terminated. The first part of this chapter 
discusses the experiment selection rule and the second section pre- 
sents the stopping and model selection rules. 

5.1 Experiment Selection Rule 

The procedure adopted for this dissertation is~ the so-called 
myopic procedure. This rule simply chooses as the next experiment 
that one. which maximizes the anticipated K-L information for the 
next stage only. - 

We assume that an upper limit, to the number of ob- 

servations is specified. This number may be infinite. An alloca- 
tion of the observations to the stages of sampling is described by 

a J xl vector n, where n. gives the number of observations 
MAX i 

at stage i. The question arises as to how the observations should 

be allocated. That is, should all J be taken at once, strictly 

rliYX 

one-at-a-time , or in different sized groups. As the first step in 
answering this, let A. denote the set of experiments in A which 


37 


38 


specify that j observations should be taken. For any given state 
k 

weQ y let a . (w) denote the element of A. such that 
3 3 

I[w,a*(w)] =■ sup I(w,a.) 

3 a , eA. 3 

3 3 

Lemma 5.1 For any w efi, and i,j such that i > i we have 
l[w,a*(w)3 21 I [w (w) ] . 


Proof ; We introduce the following notation. Let » 

5 k 

k a 1, . . .,i denote the random variables observed under (w) 

and y, (a.)» k = 1, . . . , j denote the random variables observed 
K J 

k 

under a_, , Define another experiment a^cA^ b y choosing the first 

k 

j observations according to a^ and the remaining i - j observa- 
nt 

tions according to the last i - j of a^. This leads to the ran- 


dom variables 


r * 

y k (a j> 


k ~ 1 » « • • f j 


W = < 


ek (a ^ 


k = j + 1, . . . ,1 
Because I(w,a,m,n) is positive definite and is additive for inde- 
pendent observations 


Thus 


I(w,a.,m,n) > I(w,a.,m,n) 
1 “ 3 


iCwja^) = 0 [iCwja^jmjn) ]8 

>. [I(w,a ,m,n) 3? = I(w,a*) 


But by definition I(w,a*) >_ I(w,a^) and hence 



The lemma simply proves that an experiment with more observa- 
tions will be expected to provide more information than one with' 
fewer observations. In determining an allocation one ‘should also 
consider the cost of experimenting. In particular, if we assume 
that each observation has c a constant cost associated with it, then 
it is reasonable to choose the experiment which maximizes 


I I(W ’V 


j = 1, 




MAX 


k-1 

Thus prior to stage k let m = n. 

i=l 

optimal experiment is the element a^eA 
state ^ yields 


and assume m < J,,*,,. 
: MAX 

which for the current 


The 


MAX 


i = 


,J MAX " m 


{ MAX i 

asA . J 
" 3 - 


I(w k-l ,a) | 


If sampling has not been terminated by the rules developed in 
Chapter 5.2, then we stop when £n_^ = and select the model 

according to the rules In Chapter 5.2. 

5.2 Stopping and Model Selection Rules^ 

We now discuss the problems of determining which of the postu- 
lated models is the true one and determining when the results of the 
experiments are sufficiently informative to stop sampling and make 
the choice. 

Box and Hill (1967) suggested that for their procedure, ex- 
perimenting be terminated whenever one model is clearly superior to 
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the others. This is obviously a reasonable statement but it is in 
need of formal definition before it can be used as a stopping and 
selection rule. We propose general stopping and selection rules 
and a modified version which might be used in 'certain instances in- 
volving nested models. 

(1) Stopping rule ; Let 0^ be some specified value 

1/L <6 < 1*0. Let J w “ denote the maximum number of observa- 

' m — MAX 

tions permitted o Then terminate sampling whenever either 
MAX' 

{6 } > 0 or observations have been taken, whichever 

c i=l,L i — m MAX 

occurs first* 

(2) Model selection rule : Upon termination choose the 

correct model to be H.* where 0.* = 

J J i=l, L l 

We now present a modified stopping and selection procedure 
for use with nested models which may be of some value when 0 is 
very near 1.0 and/or when is relatively large. The reason 

for presenting a modified procedure arises from the large sample 
results of Chapter 3 and the Monte-Carlo results of Chapter 6. 

First, if H * denotes the unknown true model, it is not known 

i 

whether 0 ,a . 1*0 or not. From the. Monte-Carlo results it seems 


that the typical behavior of 0^* ^ for nested models is to fairly 

rapidly increase to something near 1,0 and then fluctuate, possibly 

slowly approaching 1.0. Thus, if is very near 1.0 it may be 

that extremely large samples would be required. Thus we would like 

! 

to reduce the average sample size without seriously detracting from 

the probability of choosing the correct model. 
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To introduce the modified procedure consider the following 
example: 

V y = $x + £ 


H 0 : y - 3 1 x 1 + 3 2 x 2 + e 

If is actually the true model, then the posterior distribution 

of ( 3 ^, 62 ) under H 2 should approach a point distribution with 

0 and equal to the unknown value of the parameter. How- 
ever, 0. . may not . approach 1 = 0. Assume some small positive con- 

1 >3 

stant y is specified. Then after each stage of sampling, test if 


d = 


( 1 ) ..( 2 ) 


n 


r<2)i 2 < 

y2 j 


T 


If d <_ y then drop model 2 from contention and replace 6^ by 
0 + 6^,, Then apply the previously described stopping and selection 

rules. In this simple example, the dropping of model 2 would auto- 
matically cause sampling to be terminated. This would of course not 
necessarily be true in more general situations. 

To generalize the procedure some additional notation and con- 
cepts must be introduced. We use the symbol to denote inclusion. 

Thus H.D H. means that the model of H. is nested within the 
1 3 3 

model of The set of models (H^ is a partially ordered set 

under the partial ordering relation^. In the theory of partially 
ordered sets a chain is defined to be a partially ordered set such 
that for any two elements (H^ and H 2 say) of the set either 

H 2 or H 2 ^ . For the purposes of this dissertation we de- 

fine a string of elements from the partially ordered set as a sub- 
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set of elements such that the subset forms a chain. A maximal 
string is constructed from any string by adding all the elements 
of {H^} to the string which can be added without causing the en- 
larged set lose the property of being a chain. 

To formulate the modified stopping and selection procedure 
we first construct all of the maximal strings that can be con- 
structed from the set {H_^} and order the elements of the strings 
using the relation 3 . 

For example, suppose 0 L = 5 and the five models are as 

specified below: 

Model Model 

number equation 

1 y = B o + e i x i + e 

2 y = 3 0 + P 3 x 3 + E 

3 y - 3 q + + ^2^2 + G 

4 y - + 6-jX^ 4- + e 

5 y = s 0 + + e 2 x 2 + e 3 x 3 + e 

The maximal strings are easily verified as being 

(H X C H 3 C H 5 ) 

(h 1 ch 4 ch 5 ) 

(h 2 C h 4 C h 5 ) 

In each of these strings the maximal element is 

The modified procedure consists of computing for the first 



maximal string the squared distance of the posterior mean vector of 
the maximal element from the posterior mean vector of the submaximal 
or next largest submodel of the string. If this quantity is less 
than some prespecified value y, the maximal model is dropped from 
the set The posterior probability of the maximal element is 

added to the probability of the next element of the string until 
either only one model remains or there is no need to drop models. 
Before considering the next maximal string, all models which have 
Co been. dropped must also be deleted from the remaining strings. The 
above procedure is then repeated for each maximal string in turn. 

Once this has been completed and all models which can be 
dropped because they reduce to models with fewer parameters have 
been dropped, the same stopping and selection rules proposed for the 
non-nested case are applied. , * 

Note : The procedure just described is not necessarily the 

best or the most natural one to use for combining models. For ex- 
ample, an alternative to the distance of the means, might be to com- 
bine models when the probability distribution of the maximal element 
is sufficiently concentrated about the mean of the submaximal ele- 
ments This would have the advantage of using the information con- 
tained in the precisions of the distributions also. 



CHAPTER 6 


COMPUTER SIMULATION RESULTS 

The purpose of this chapter is to report and discuss the re- 
sults of a number of Monte-Carlo simulation studies of the sequen- 
tial procedure proposed in Chapter 5. The chapter is divided into 
four major sections. The first section describes the general simu- 
0 lation procedure and presents a brief description of the algorithm 
used. A computer program based on this algorithm is described in 
further detail in appendix A. The second section presents and dis- 
cusses the results of a number of simulations performed to gain fur 
ther information about the large sample behavior first discussed in 
Chapter 3. The primary concern is with the posterior probabilities 
and the posterior means of the parameter distributions after a larg 
number of observations. The third section presents and discusses 
the results of some simulation studies -of the proposed sequential, 
procedure when the number of observations is limited and when the 
stopping rule of Chapter 5 is used. The primary concern is with 
the probability of the procedure actually selecting the correct 
model and the average sample size required until termination. The 
last section of the chapter presents an overall discussion of the 
results. 

6.1 General Simulation Procedure 


The sequential procedure proposed in Chapter 5 consisted of 
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(1) an experiment termination rule, (2) an. experiment selection 
rule, and (3) a model selection rule. Because of the mathematical 
complexity of the posterior distributions involved it was not 
feasible to analytically examine how well these rules work. The 
general procedure by which the Monte Carlo simulation technique 
was used to study performance is outlined in the following algorithm 
lo Input : 

li n the prior means of the parameters of the models 
q the prior precision matrices of the parameters 
of the models 

0^ q the prior probabilities of :the models being 
correct 

N the number of simulations 

0^ stopping probability ■ 

J...„ maximum number of observations 
MAX 

i the model chosen to generate the observed variable 

y values of the parameters of the true model 

2. n *■ 0 
3 * PCS + 0 

4. N j , -e 0 (for i = 

5. j 0 

6. j + j + 1 

7. Determine optimal aeA as described in Chapter 4. Denote 

* * 
as a and let M * denote design matrix for model i 

a 

when a is chosen*, (All simulations in this dissertation 
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consider strictly one-at-a-time sampling for simplicity.) 

8. . y M *y * 

J a 

9. Generate a pseudo -random observation e . from a N(0,t) 
distribution. (Described in detail in appendix A) 


10. y . ■<- y . + £ . 

J "J J 

11. For l - 1, 
y. and 6 


,L compute 0 ¥ . , and p\ . from 

X »J X- ,J 


Ilf 

\,j-l’ and y £,j-x as described in 


Chapter 3 


12 0 Find k such that G, , * MAX{8 } 

± i,j 

13 . If j >_ J 


MAX ° r ^k,j — t0 ^ * Otherwise go to 6 

14. N, <- N.- + 1 

j 3 

15. If k « i*; PCS +■ PCS + 1 

16. n ■+■ n + 1 * 

17. If n >_ N go to 18. Otherwise go to 5. 

18. PCS + PCS/N 


19. ASN +• 


> J MAX V 

IS "> 


20. Stop 

Upon stopping, the value of PCS is the observed probability 
of correctly choosing i* as the true model for the prior distribu- 
tions specified when in fact the true value of the parameters is 
* - 

given by p . ASN gives the average sample number upon termination, 
The above algorithm can be easily used for either large sample 
or small sample studies. For example, for large sample studies set 

G m ~ 1 ‘ 0, N " and J MAX to some large number, say 100 or 500. 
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For small sample studies set 6 < 1*0, J,,.„ to some small number, 

v m MAX 

and N to some larger number, say 500 or 1000 * 

6.2 Large Sample Studies 

In this section we examine the large sample properties of the 
posterior probabilities of the models and the posterior means of the 
parameter distributions. Three sets of problems are studied. First, 
two sets of nested polynomial models are studied. The posterior 
probabilities of each model, the posterior means of the parameter 
distributions, and the proportion of times each of the allowable 
values of the independent variable is 0 chosen as optimal are tabu- 
lated for simulations of 100 and 500 observations. Second, one set 
of nested factorial models is studied for thrte different prior dis- 
tributions’ on the models. And third, one set of non-nested fac- 
torial models is simulated. For the last two, the posterior proba- 
bilities and means of the parameter distributions are tabulated. 

6.2.1 Polynomial Model Studies 
Two sets of nested polynomial models are considered which 
have the following general form: , 

£-1 

V y - * = i.l 

£ jsb 1 

Two values of L are studied, and for each of these choices, two 
choices of H.* are made. The values of t, 8. and _ are 

l £,0 £ , 0 

specified as 

t = 100.0 

V . 

£,U 


T 
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for all simulations. The values of ^ are tabulated at the tops 
of figures 1 and 2 and the resulting functions are graphed on the 
interval x [-1,+1] at the bottoms of the respective figures. For 
L = 4, the two choices of are and • For L = 6, the 

two choices of are and H^., For simplicity, the actual 

values of the parameters used to generate the data were chosen to 
be y^A q for each of the four cases. 

For these simulations, the definition of A was arbitrarily 
taken to be 


A = {a (l) ; i = 0, . . .,9} 


where 




Note that sampling is strictly one observation per stage. 

The simulation results are summarized in table 1 and given in 
further detail in tables 2 through 9. For each choice of L and 

A 


i , five simulations of 100 observations and five simulations of 
500 observations were performed. For these simulations, the sample 
paths of the 6 0 , were printed out and the choice of at each 

stage were printed. The posterior means of the parameter distribu- 
tions were printed only after the last stage. Tables 2, 4, 6, and 8 
give the posterior probabilities after 100 observations and the 
first 100 out of 500 observations. The proportions p of using 


( 1 ) 


ire also given. Tables 3, 5, 


7, and 9 give the same informa- 
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MlO= (-1) 



Figure 1. - Tabulations of the prior means of the parameters 
and graphs of the resulting functions over the interval 
[-1, +l] for large sample polynomial study one. 
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Figure 2. - Tabulations of the prior means. of the param- 
eters and graphs of the resulting functions over the 
interval -1, + l] for large sample polynomial study 
two. 
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tion for the 500 observation simulations. 

Figures 3 and 4 present typical sample paths for the posterior 

probability of the correct model. In figure 3, the value of ®2,j 
is plotted for the first 250 observations of the third simulation 
for L - 4 and i* - 2. In figure 4, the value of 83 is 
plotted for the first 250 observations of the first simulation for 
L = 4 and i* - 3. These figures illustrate the typical behavior 
of 6 .* . It fairly rapidly rises to a value of about 0.85 tp 

0.95 and then slowly and erratically oscillates. This is suspected 
to be because of the nested nature of the model equations. It was 
because of this behavior that the modified selection rule of 
Chapter 5 was first introduced. Consideration of the posterior 
means of the parameter distributions will also provide some infor- 


mation concerning this modified 

For L « 4, consideration 

I 

as j increases, 
with the conclusions of Chapter 



rule.. 

of tables 2 , 3, 4, and 5 show that 

I 

for i > i*. This is in accord 

- * 

3. For L - 6 and i = 3 we 


again see the same close’ agreement with Chapter 3 as evidenced by 
tables 6 and 7. However, for i* = 5, an entirely different situa- 
tion arises. To understand this we should note that the model used 
to generate the sequential observations is 

' 4 

y « 0.5 x + 0.1 X + e - 

• c- 

This function can be very closely approximated by a model of the 


y 


ax 


+ bx + 


form 
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over the range of x values considered. And in fact we note that 

there is a marked preference for choosing the lower degree model as 

indicated by G- , becoming close to 1.0. It is also interesting 
d »3 

to note the behavior of u. , for i > 3. We do not see that 

i,j 

*3 j\ 

)J J as might be expected when is so close to being 

ft / 




true, except for the case of 1=4. For we note that the 

3 

average posterior mean of the coefficient of x is quite close to 

2 

zero and the sum of the posterior means of the coefficients of x 

/j. 1 ’ —V 

and x is quite close to 0.1. For p^. we note that the sums of 

2 4 

the posterior means of the coefficients of x and x is close 

to 0.1 and the sum of the' posterior means of. the coefficients of x, 
3 5 

x , and x o is close to 0.5= From these simulation studies it is 
not clear whether this behavior is simply because 500 observations 
is not. a sufficiently large number to discriminate well between such 
nearly equivalent functions or if this behavior will persist no 
matter how large the number of observations; 

We now turn to a discussion of the observed proportions of 
times the were chosen as the optimal experiments. From 

tables 2 and 3 which present the results of L - 4 and i = 2 we 
see that the largest p^ are for p^, p^, and p^ . These cor- 

respond to x = -1, x = -1/9, x = +1/9, and x = +1. Because of 
the discretization of the interval (-1,+1) we might assume that the 
asymptotically most informative experiments were x - -1, x =0, 
and x = +1. From tables 4 and 5 we see the largest p are p„, 

i 0 

p£, p^, and pg corresponding to x - -1, x ■- -5/9, x -• +5/9, and 
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x - +1. The relationship of these proportions and x points to the 
experimental designs which are optimal from other considerations 
might be interesting. For example, Kiefer and Wolfowitz (1959) con- 
sider optimal designs for regression problems ' of a somewhat differ- 
ent nature. The comparison of the current results with such other 
works is currently being pursued but will not be reported in this 
dissertation* 

6*2.2 Nested Factorial Models 

A second set of simulation studies were made using the follow- 
ing models 

H r y - e 0 + « 

H 2 * y = p o + h x i + £ 

: y = Bq + + & 2 X 2 + e a 

H 4 : y = + ^l x l + ^2 X 2 + ^3 X 1 X 2 + £ 

sk -f 

with t = 2 * 0 , Q - I, and i =3. The prior means y^ ^ were 

chosen as 

“l,0 - 
y 2,0 = ( ° 5i) 
y 3 ,0 ~ “(p / 

y 4,0 = 

Three sets of 0^ ^ were chosen: 



55 


1 . 6 


1,0 


2,0 


3.0 


4,0 


0.1 

0.2 

0.3 

0.4 


°^0 & °' 25 1 


« 1,4 


3. e 1>0 “ °*4 

6 2,0 " °‘ 3 
e 3 , 0 - 0.2 


0 4,O ^ °‘ 1 


The experiment space A is defined as A = {(x^x^): x^ = ±1). 
Note . that experimenting is strictly one~at-a-time 0 'The sequential 
selection procedure of Chapter 5 was used for five simulations of 
500 observations each. The results are presented in table 10, We 
note that the posterior values are again in close agreement with 
Chapter 3 and the results of the polynomial models. There does not 
seem to be a pronounced effect upon the posterior probabilities of 
the models from changing the prior distribution although there does 
appear to be slightly higher posterior values of 8^ when the 

0 _ distribution is skewed toward the lower values, A possible 

Ar y - 

explanation for this is that when this distribution is skewed toward 
the high values, the procedure is choosing experiments primarily to 
discriminate between and H^. Since is true, the model of 



56 


will rapidly become close to that of and the resulting ex- 

periments will not be very informative. When the prior probabili- 
ties are larger for the lower degree polynomials, however, the pro- 
cedure chooses experiments primarily to discriminate between 
and H^. These experiments should then more rapidly tend to prove 

and to be inadequate, 

6.2,3 Non-Nested Factorial Models 
In this study, the following non-nested models were studied, 

o 

H r y " 6 o + s i x i + S 2 X 2 + E 

H 2 : y - + B 3 x 3 + e 

H 3 ; y “ + P 2 x 2 + ^3 X 3 + c 

The values of the parameters are chosen as 



t = 0. Of, 1,100 

7 * = 


The experiment space A was assumed to allow only one observation 

at a time with x_^ - ±1. Five simulations were performed for each 

value of Tc For t = 100 it took only three observations for 0 
• * . 

to become 1.0 (within the accuracy of the computer). For x - 1.0 
the number of observations required for the final posterior proba- 
bilities to reach 1.0 are tabulated in table 11. For x - 0.01, 



57 


1000 observations were taken and the resulting posterior probabili- 
ties are given in table 11. Again “the results are in general agree- 
ment with Chapter 3. 

6.3 Small Sample Performance Studies 


In this section we examine the performance of the proposed se- 


quential procedure as measured by the PCS and ASN values . First, 
two studies are presented of the problem of discriminating among the 
three models 


V y - Vi + e 

V y - P 2 X 2 + E 

H : y = 6 X x + e 2 x 2 + E 

The first study assumes is true and the second study assumes 

is true. The experiment space A is defined as 

A = {(x^jX^): = ±1; one-at-a-time sampling} 

i 

Then we consider the problem of choosing among the four nested 
models. 


H i : y = B 0 '+ e 

H 2 : y « 6 q + 6^ + e 

H 3 : y = P 0 + e i x l + 6 2 x 2 + E 

V y ” 6 0 + e i X l + 8 2*2 + 8 3*l x 2 + £ 

where A is as in the first two studies. 

The primary reasons for concentrating on these simple models 


are that the parameter spaces are of a low enough dimension that 
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they can be visualized and they are small enough that extensive 
simulation studies would not require inordinate amounts of computer 
time. The fact that the parameter spaces can be visualized allows 
the effect of varying prior means upon PCS and ASN to be more easily 
grasped. Note that the modified stopping and selection rule was 
not used. 

6.3.1 Study One - Assumed True 
We study discriminating among 

H i J y “ ^l x l + z 

H 2* y " e 2 X 2 + E 

: y = + + E 


2 2 


A = {(x^jx^): - ±1; one -at -a -time sampling) 


, > 

where 


y =1 

£,0 


1,0 


1,0 


- ( 1 . 0 ) 


p 2>0 (1 * 0) 


and 



Then a number of simulation experiments were performed for 
each combination of: 

T = 0.50, : 1 o 0 , 2.0 

6 = 0.70, 0.80, 0.90 

m 


0 



m 3,0 

The experiments for J. 
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0 

♦ 

0.0\ 

(Q.5Q\ 

/1.0\ ll,S\ 

1 * 

[ | 

’1 M j 

OcO / 

^ 0c 50j 

\l.0j \i.5j 


MAX 


8 used 1500 simulations and for 


1,,„ = 16 used 1000 simulations. 

MAX 

The choice of prior means deserves some comment , Figure 5 

illustrates the points in coordinate space corresponding to 

the prior means. The points corresponding to p^ ^ and p ^ q are 
-v* -r 

as close to p as possible since p^ ^ is restricted to the hori- 


zontal axis and p^ q to the vertical. The four choices for p^ q 

then span a range about p and hence the resulting PCS and ASN 

values will indicate the importance of mis-specif ied prior means. 

Tables 12 and 13 present the observed PCS and ASN values for 

the combinations of 6 , x, and p 0 n . These results are also 

mr ’ 3,0 

plotted as parametric surfaces in figures 6 through 9. 

In general, the results are about what should be expected. 

The PCS increases with x and ASN decreases with x* PCS increases 
~y 

as p^ q gets closer to p We also note that in most cases, PCS 

increases with 6 for fixed x and vu There is, however, a 
m 3,0 

distinct dropping off of PCS v;ith 0^ along the peaks of the sur- 


faces, There does not seem to be any ready explanation for this. 
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Figure 5. - Illustration of prior means for 
performance simulation experiment one. 





m 
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6.3.2 Assumed True 

A much less extensive study of this case was made than the 
case of assumed true. The same model equations were postulated 

and we assume 




Si.o ■ (0 - 0) 
v 3 0 = (0.0, 1.0)' 

V* = (1.0) 



The values of t, 0 , and p 0 ~ which were simulated are tabu- 
la 2,0 

lated in table 14 along with the simulation results . Figure 10 
illustrates ..the prior .means * Only.. one level of ( = 8.) ' was 

considered* Also, only 500 simulations were performed for each of 
these cases. The PCS results are also graphed as a parametric sur- 
face in figure 11. The results are generally the same as for 
true c 



Figure 10. - Illustration of prior means for small 
sample performance simulation experiment two. 
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6.3.3 A Four Model Problem 

In this section wc study the ability of the sequential pro- 
cedure to choose the correct model from the following set of com- 
pletely nested model equations. 

V y = 8 0 + £ 

h 2 : y = 6 o + 6 i x i + c 

H 3 ' y = B 0 + e i X l + B 2 X 2 + E 

H 4 : y = 6 0 + B 1 X 1 + B 2 X 2 + B 3 X 1 X 2 + E 

The prior distributions are defined by 



The equation used to generate the observations was that of vith 

values of the parameters given by 



The value of x used was 1.0. 

For fixed values of 8^ , the values of the prior parameter 

means for and and the four prior means for can be 

plotted in 3-space as in figure 12. For each of the four choices 

of p- A , three values of 0 (=0.7, 0.8, 0.9) for = 8 were 

1 , u m max 

used and the resulting PCS and ASN values for the 12 combinations 

~ *. i , . . , ; ^ t • i r ■n/ - ' r> t . , -i * . i ; „ r ? . ? *y / - 


cases, 1000 simulations of the procedure were performed. 




^ 2, 0 


<(0, . 5, 0) 


po = constant 




gure 12. - Illustration of the prior means when P 3 is held 
constant. Small sample performance simulation experiment 
three. 


M 3 . 0 • < 0 . 1 - U 

h 3 g = (0, .5,0) 


0 M 3 0 = U.0, .50, 0.0) 
, M3 0 =(0 '°> 


Figure 13. - Probability of correct selection (PCS) as 
a function of 8 m and 1 X 3 q for J^ ax = 8 and R 3 
true. Small sample performance simulation experi 
ment three. 
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6.4 Discussion of Results 

We now make some general observations concerning the results 
of the simulation experiments. 

First, consider the large sample results. In the context of 
the fact that sequential procedures are primarily developed in the 
hope that reliable decisions can be made with small samples rather 
than large samples, these results are not of primary importance. It 
is interesting and informative to know, however, that the procedures 
are consistent. Since the study of limiting posterior distributions 
resulting from sequentially chosen experiments is known to be an 
extremely difficult and delicate problem, simulation experiments may 
be helpful by indicating to researchers what large sample behavior 
is likely to be true. In the problems studied in this paper it 
seems quite likely that when non-nested models are encountered, the 
posterior probability of the true hypothesis has a limiting value 
of unity. It also seems most likely that the limiting posterior 
mean of the true hypothesis does indeed equal the values of the un- 
known parameters generating the data. 

When nested models are encountered, however, the results are 
not as enlightening. It appears that if the posterior probability 
of the correct hypothesis does not achieve a limit of unity, it at 
least attains a large value and then randomly fluctuates about that 
value. There is indication that the conjecture of Box and Hill 
that for these nested models there is a distinct preference by the 



sequential procedure to choose the model with the smaller number of 
parameters as true. For instance, the polynomial study L = 6, 
i =5 indicates that if a model with more parameters is true 
but can be approximated closely by one with fewer parameters, 
there is a preference for the smaller model. 

In examining the small sample performance simulation experi- 
ments, it is seen that PCS drops off fairly rapidly as the distance 

o 

of the prior mean of the correct model from the true values of the 
parameters increases. This supports the conjecture of Chernoff and 
Meeter et al. that there may often be "initial bungling." It should 
be noted, however, that in all cases . studied , the prior means of the 
competing models were all set to be as close to the true model 

parameter values as could be done. Thus, in a sense, these experi- 
ments can be considered to be presenting the most unfavorable situa- 
tion possible to the sequential procedure. In actual application it 
might be more reasonable to assume that the prior distributions of 
all the models are mis-specif led to the same extent. This problem 
of "initial bungling" should also indicate that the statistician 
should have the prior precision matrices of the parameter distribu- 
tions be as vague as the prior information permits. 

One approach studied by Kiefer and Sacks (1963) was to plan 
small initial experiments as a basis for gaining information to plan 
a large second experiment. An alternative not studied in this dis- 
sertation, but which seems worthy of investigation, would be to set 
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a lower limit, say J as the minimum number of observations 
taken before a stopping rule is applied. The sequential procedure 
would use the same rule as developed for selection of experiments 
but large posterior probabilities on the models would be ignored 
until a sufficient number of observations are taken to avoid the 
consequences of initial bungling. This also makes sense from the 
point of view of obtaining parameter estimates. Surely an experi- 
menter wculd not be content to terminate sampling with two or three 
observations even if the resulting probabilities are overwhelmingly 
in favor of one hypothesis unless he had extremely good prior infor- 
mation. 

The last topic to discuss is the modified stopping and selec- 
tion procedure introduced in Chapter 5. This was not' applied to any 
of the simulation experiments performed in this dissertation. The 

large sample simulation results indicate that when 6 and/or J 

m MAX 

are large, then this modified procedure may be of value. For the 
problems considered here, it is seen that even for nested models, 
the unmodified procedure performs quite well when 

MAX 


is small. 



CHAPTER 7 


EXAMPLE OF APPLICATION 

This chapter first presents a general outline of the situa- 
tions in which the results of this paper may be applied. Following 
this an example from the literature is presented « The purpose of 
this example is to illustrate how the information available from 
previous experimentation can be translated to the information re- 

c 

quired for the application of the sequential procedure developed 
herein,. 

A Bayesian framework is used in this paper because in a great 
many applications there does exist some prior information which can 
be incorporated. The Bayesian approach to statistical inference is 
the most natural and satisfying method of incorporating prior knowl- 
edge. This prior knowledge may arise in several ways. 

For example, when expensive or large experiments are contem- 
plated, there is often available data from pilot studies, the liter- 
ature of the field, or poorly designed prior experiments* Typi- 
cally, some type of regression analysis is performed on this data 
but there is so little data that practically no conclusions can be 
drawn, only recommendations for further experimentation. The re- 
sulting equations, however, provide a very convenient starting point 
for the application of Bayesian methods. 

In other situations, an experimenter has a great deal of ex- 


73 



74 


perience in experiments that are similar and involve factors with 
which he has some previous experience,. In these cases it may be 
safe to extrapolate his acquired knowledge from the similar but 
different experiments to the current experiment. If so, this may 
indicate some characteristics of the model equation. 

A third possibility might arise for example in the carrying 
over of laboratory results to a production process or out-of- 
laboratory process o In the laboratory greater control can be 
exerted on many variables and typically only a small number of 
variables may be investigated* Often one or more mechanistic models 
are available* When the process is taken out of the laboratory, 
there will be less control over other variables and they must be 
accounted for by adding them to the model* Thus the experimenter 
is faced with the situation of having a partly mechanistic model 
and a partly empirical model. . If the mechanistic model is suffi- 
ciently smooth in the region of interest, factorial or polynomial 
models can be applied in these cases and prior information might 
indicate which interactions or terras are most likely to exist. 

The example we consider is studied in Lloyd and Lxpow (1962) 
and Draper and Smith (1966) . In these books the data presented in 
table 16 is used to illustrate some topics in the design of experi- 
ments and multiple linear regression analysis* The dependent vari- 
able y is the chamber pressure in rocket engines put on test. 


The four controlled variables are 
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= temperature of cycle (starting) 
z - = vibration level 

^ 4 

z ^ = shock by dropping (temperature) 
- static fire temperature 


We first postulate the model equation given by equation (7-1) . 
y = S 0 + h z l + 6 2 z 1 + 6 3 z 2 + e 4 z 2 + B 5 Z 3 + 6 6 z 3 + S 7 Z 4 + B 8 Z 1 Z 2 


+ B 9 Z 1 Z 3 + B 10 Z 1 Z 4 + e il Z 2 z 3 + 6 12 z 2 z 4 + B 13 Z 3 Z 4 + E 


(7-1) 


The results of a multiple linear regression analysis of the model 


are summarized in table 17. The terms of the model are ordered in 
table 17 in decreasing order of descriptive significance level. 

The experiment is highly saturated with respect to equation (7-1) in 


the sense that 14 parameters are estimated from the data from 
18 distinct combinations of levels of the independent variables. 
There are also quite a few high correlations among the terms of 
equation (7-1) and hence high correlations among the estimated pa- 
rameters. The power of the resulting t-tests may be somewhat low 
under these circumstances.. From examination of the various descrip- 
tive significance levels, the model equations tabulated in table 18a 
can be considered reasonable. The prior means of the distributions 
are also given in table 18b. The prior precision matrices may be 
derived by multiplying t times the submatrices of order 3, 6, 9, 
and 15 of *the matrix given in table 18c. How. these prior distribu- 


tions were determined is now described. 

Tn multiple linear regression, under the usual normality 



assumptions, the parameter estimates from the model 




+ e 


are given by 


t = (m'm) 1 M , y 


and we know 

I - n(|, —y m'm) 

0 

o 

Thus, for the first three models of table 18, the prior means and 

-v 1 > 

precision matrices would reasonably be the $ and — ^ M M derived 

o 

*<by least squares analysis using the appropriate subset of data from 
table 16 . This is hew the values of q* \ 0 for ^ 
were derived. For 1 = 4 and the data of table 16, the full equa- 
tion is not estimable because there are not three levels of z^ to 

2 

estimate a coefficient of z^ Thus least squares estimates were 
computed and m'm computed for the first 14 terms of model 4, Then 
an essentially diffuse prior was specified with respect to by 

setting the prior mean to zero and adding the last row of the matrix 
in table 18c to m’h. The diagonal term was arbitrarily chosen to 
make the matrix nonsingular yet nbt comparable to any of the other 
diagonal elements in magnitude. 

To complete the information required, x must be specified and 

2 

0 n chosen. From the data of table 1.7 an unbiased estimate of o 

X,j u 

is 1.85 as computed from the replicated points. Thus we may use 

t » y^g5 = ^.541. To deterraine 0 £ 0 it: be to 

examine the F-ratios for lack-of-fit for the first three models in 
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table 18. These are F - 5 <295, F = 2.038., and F = 1.871. These 
statistics are significantly large at about the 0.90, 0.80, and 0.75 
levels, respectively. Based upon this, the following values of 
0 n seem reasonable 


0 

e 


1,0 

2,0 


= 0.10 

= 0.30 


0 


3,0 


0.30 


0, n = 0.30 
4,0 


4 


O 
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APPENDIX A 


COMPUTER PROGRAM FOR SIMULATION STUDIES 


The general flow of operations and computations performed by 
the program is described by the algorithm given in Chapter 6,1. The 
input required to perforin these computations Is first described. 

How these computations are achieved is described briefly by giving 
the major functions of the subprograms constituting the complete 
program . A complete FORTRAN listing is given. 

INPUT 


The program, as presented here, can only accommodate poly- 
nomial models over the interval [-1,4-1] and two-level factorial 
models o This can be changed by writing one new subroutine (MFORM) 
to handle more general models e The program identifies the param- 
eters by their integer subscripts and computes the x_^ values for 
the M-matrices according to the following convention: 

1. For polynomial models, the subscript i indicates the 

i 

parameter which is the coefficient of x . 

2. For factorial models, the coefficients are assumed ordered 
in the standard order according to the description in Sidik and 
Holms (1971). The treatment combinations are also assumed to be in 
standard order and the independent variable values are constructed 
as described in Sidik and Holms (1971). 

3 . The order of the models as specified for input are written 
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such that the parameter subscripts are in increasing order. 

The specific input cards are now described below and illus- 
trated by the input for a case run in Chapter 6.-3. 1. The problem 
input is specified by the models and parameter values : 

H 1 : y = 3-jX^ + c 

h 2 : y = e 2 x 2 + e 

: y - + ^2 X 2 + £ 


V 


l 

0 - 0.70 

m 

e fL,0 

1 

3^ 

/£ = 1,2,3 

t - 0.5 



J MAX = 8 



1500 simulations 



Start random number generator with 041 574 501 221 c 
h IDENTIFICATION (13A6) 

One card for Hollerith input description of rhe problem. 

2 . NAMELIST INPUT ($NAML) 

Most of the control parameters are included in a NAMELIST 
input set. The list of parameters and their purpose fol- 


lows : 
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NFAC For factorial problems, this supplies the number 
of factors* For polynomial problems, this vari- 
able need not be supplied. 

NHYP Number of hypothesized models* 

NSYM Number of simulations to run,. 

MXTRTS J WAV 
MAX 

NTREND Not used for this report. Set to zero. 

LEVOUT An output control variable* Certain basic out- 

C' 

put is ° automatically printed * Extra intermedi- 
ate output can be. printed by setting LEVOUT to 
an integer between 1 and 7* For performance 
studies set to 0* . For large sample runs set to 
2= For debugging set to 7. 

' 'it 

LTRUE Supplies i , the correct model subscript. 

TAU T 

CSTOP Q 

m 

TMaX An upper limit on execution time* If this limit 

is exceeded, t.he program dumps for a restart. 

IFSTRT Set T for supplied starting value, for random 
number sequence. Set F if sequence is to 
start with initialization value. (See descrip- 
tion of subroutines .RAND and SAND for further 
information.) * 

RESTRT Is this problem a restart, of a case, terminated 
by exceeded time? T or F 
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POLY T implies polynomial model. F implies 
factorial model 

NX For polynomial models, the x space is re- 

stricted to the interval -1 to +1. NX speci- 
fies the number of points used to discretize 
the interval into equal increments. 


3. FORMAT FOR PRECISION MATRICES (13A6, A2) 

For each set of model equations supply one set of 4A, 4B P 
4C, 4D, and 4E» 


4A 0 NUMBER OF PARAMETERS IN MODEL, PRIOR PROBABILITY OF 
MODEL (16, F12.6) 

4B. PARAMETER SUBSCRIPTS (1316) 

4C. PRIOR MEANS (12F6.0) 

4D. TRUE VALUE OF PARAMETERS (12F6.0). 


This card should be supplied only for the set corresponding 
to the correct model. 

4E. PRIOR PRECISION MATRIX 


Only the upper triangular half of ¥ is specified with the 
order being 


u> u> v etc 

11* 12’ 22* y 13 j 


5. STARTING VALUE FOR PSEUDO-RANDOM SEQUENCE (012) 


A graphical illustration of a data deck is given in figure A1 
and a FORTRAN sheet giving the sample input is given in table Al. 
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ONE SET FOR 
EACH MODEL < 
EQUATION ! 


Starling value for random 
sequence 1012) 


@ 


r~ 



d) 



A - This card is optional and its presence depends upon NAMELIST data. 
B - Supplied only for true model. 

Figure A-l. - Graphical illustration of card input for simulation program. 
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SUBPROGRAMS AND THEIR MAJOR FUNCTIONS 
lo SEQDES. This is the main program. It reads the input 
cards, exercises general control over the other subroutines. It 
also outputs the final results. 

2. ACOMB. Called once by SEQDES at the beginning of each 
case. This subroutine scans the lists of parameter subscripts for 
each model and constructs from them a new lis t of subscripts in 
ascending order which contains the parameters appearing in at least 

o 

one model. 

3. SVSTRT (GTSTRT) , A double entry subprogram. The. entry 
SVSTRT is used once at the beginning of each case to save the prior 
probabilities, means, and precision matrices a Then after each sim- 
ulation, entry GTSTRT is used to re-initialize the x^orking proba- 
bility, mean, and precision vectors to the original values „ 

4* MFORM. There are two versions, MFORMX and MPOLY* The 
first is for factorial runs and the second for polynomial models. 
These subroutines construct the M-matrices required for each model 
\fhen given the experiment choice (a level of x for polynomials, 
a treatment combination for factorials). It is called by INFX 
and YGEN. 

5. INFX (UPDATE). Entry INFX accepts the current state of 

sampling as defined by {8,, . , v, . , 'T ; ; i. - 1,L] and determines 

» J x-O , j 

the expected K~L information for a specif ied - experiment , Repeated 
calls then are made by SEQDES to determine the optimal experiment. 

J-iiiCty LiiuniL i _a.-i--i.tzu uy iuuu auU at-vc^'-c .iie v. u . luUc c . a t- v. 
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and an observed random response to compute the posterior distribu- 
tions * 

6. RANDUM. Called once for each random observation taken. . 
Generates a random observation, e, from N(0 ,t). 

7. YGEN. Called once for each random observation. This rou- 
tine accepts the N(0 ,t) variate generated by RANDUM and calls MFORM 
to compute M, where M is the design matrix appropriate for the 
correct model and the experiment chosen as optimal. Then 

y = Mp -he 

c a 

is computed. After generating y, posterior values for 6 are 

computed. Sampling for that simulation trial is then terminated if 

any. 0„ exceeds 0 . If sampling is not terminated, YGEN calls 
. a m 

UPDATE to compute the posterior y (y) and ¥^(y), and the sampling 

procedure is continued. 0 

8o COUNTX. This subroutine is called by YGEN whenever a 

0„ > 0 or by SEQDES whenever observations have been 

i — m J MAX 

taken* It counts the number of times each model is chosen and re- 
cords the distribution of sample sizes. These are then output 
after all simulations have been computed. 

9. INVXTX. Inverts a symmetric matrix stored in the lower 

1 ) 

symmetric storage mode. 

10c TRIOUX* Outputs a lower triangular matrix. 

11. MTVEC* Computes the product vector resulting from the 
multiplication of a vector and a symmetric matrix. 
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12 o RAND (SAND). This pair of entries provide the pseudo- 
random sequence of uniform random variates . SAND is an initializ- 
ing entry which must be called before any calls to RAND. When RAND 
is called, it computes the next value in the random sequence from 
the current value , The return argument of RAND is the floating 
point uniform random variate* The input and retm-n argument of 

o 

SAND is an argument which saves the integer value of the random 
variate o The generator is the multiplicative-congruential type 
^obtained by taking the low order 36 bits of the product r 
where 

r ^ is the previous random number - 

Tq is 1 and 

15 

k is 5 

The properties of this generator are discussed by Taussky 
and Todd (1956) and Coveyou and Macpherson (1967). 

13* BCREAD (XI, X2). BCDUMP (Xl,X2). These routines, respec- 
tively, read and punch cards in absolute binary at the rate of 
22 words per card. The data read or punched begins at the location 
in core of the variable XI and ends at the location in core of the 


variable X2 = 



B9 


1 


OCOOtOG COMHDN/UNI TS/ I UNf N » I UNOUT. , MASK, LEVOUT.PFMT (4 ) 

00002 00 COMMON/ XR AND/ Y f EP$,UNIF,U 

00C0300 COMMON/ERR/T AU , S l GMA 

00C0400 COMMON/ A L PH / I AL PH( l 0001 , I I A1_PH{ 51? ) , ALPHMU (1000) i R E ALMU ( 5 12) 

0000500 X .NPARAMUO) tKUD( 11 I ,PKU( 1000) 

0000600 COMMON/ PS /PI NOEX< LI ), PR EC (2 000) 

0000700 INTEGER PINOEX 

OOOCSOO COMMON/CNTRLS/NHYP.NALL .NFAC ,NFULL, NFULM1 , TRTMNT , NT RENO, NOR St (.TRUE 

00CC900 INTEGER TRTMNT 

OOCIOOO COMMON /PRF MNC/NA VG ( 1 0 00 ) , PC S , X MSE , X MS T , I N I T AL , IHCNT(IO) » MS OTAR 

0001100 COMMON/ 1 NF / X INF 

0001200 COMMON/PROBS/THETAI 10) ,CSTOP 

0 001 TOO C 

0 001 AGO C 

0001500 C 

0CC1600 C0MM0N/MM/0ESM( 1000) 

0001700 COMMON /YDT ST / S ( 1 0 > , ft t 1 0 t , SV I 10 ) , R V < 10 ) 

0001800 COMMON /X TR A/ AA( 1000 1 , B( 2000) ,C< 2000 ) , f)< 1 0 ) 

0001900 C 

0002000 C** ** ** 4$ • • 4 * 44 4;* 44 4* 44 ** 4 4 44 44 

0002100 C 

0002200 OATA EN0CRD/4HENDCRD/ 

0002300 REAL 10ENT 

00C2A00 EQUIVALENCE ( X I , I X ) , ( K S , Sft ) 

0002500 DIMENSION I0ENT<14) 

0002600 DIMENSION INPFMTU4) 

0002700 LOGICAL 1 F S T RT , RE S T R T 

00C2000 LOGICAL KPGEN 

0002900 LOGICAL POLY 

0003000 N'AMEL I 51 /NAML/NFAC , NHY P , N S Y M , MX T RT S , NT RENO* L EVOUT , L TRUE i T AIJ, 

0003100 X CSTOPt TMAX, IFSTRT,RESTRT 

00C3200 X , POLY, NX 

0OC3400 C 

0003500 c************* 4444444444444444444444*444444444444*444444444444444**44444 

0003600 C 

0003700 KPGEN=. FALSE. 

0003800 iGoro=i 

00C3900 CALL TIMEllTSTRTl 

00C4000 1 READUUNIN,50A0> IDENT 

OOOAtOO IF! lOENT(l).EO.ENDCRD) STOP 

00CA200 WRl TEC IUNOUT, 6010) I DENT 

0004300 C 

0004400 READ! I UN I N « NAML 1 

OOC450O WRI T£ ( IUNOUT , NAKL ) “ 

00C4600 NEULL=2**NFAC 

0004700 NFULM1=NFULL-1 

0004800 IF(PDLY) N^ULL=NX 

0004900 S I G M A = 1 .0 / SQR T < T AU ) 

00050 CO READ! H/NIN, 5040 ) INPFMT 

00C5100 MUO ( I ) - 1 

00C8200 • PINDEX(l)=l 

0C05300 DO 190 N= 1 , NHY P 

00C5350 I HCN T < N ) =0 

0005400 WR i TF.< IUNOUT. 6365) 

0005500 READ! IUN IN, 5020 ) NPARA M t N ) , THET A ( N ) 

0005700 MUO( N*l > =M1I0 (N » «• NPARAM(N) 

0005800 P I NO EX (N*l 1 =PI NDEX I N) + ( NP AR AM I N ) 4 ( NPAR AM ( N ) ♦ 1 I) /? 

00C590O MULO = 'RIO(N) 

0006000 HUH I = MUO(NM)-l 
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onc6i oo 
0CC42 00 
0006300 
0 0064 00 
90C6500 
0006600 
0006700 
0006800 
0006900 
0007000 
0007100 
00C7900 
OOCROOO 
0008200 
00CR300 
0008400 
0008500 C 
00 08600 
00 08 7 CO 
00C8800 
0008900 C 
0CC9000 c 
0009100 C 
00C9200 
00C9300 
0 0 09 4 00 
0CC9500 
0CC94O0 
0009700 
0009800 
0CC99OO 
0010000 
0010100 
0010200 
0010300 
0010400 
0010800 
0010600 
0010700 
0010800 
0010900 
0011000 
0011100 
0 0112 00 C 
0011300 C 

0011400 r. 
0011800 
0011600 
0C1 1 700 
0011*00 
0011900 
0012000 
0012100 
0012200 
0012300 
0012400 
0012800 
0012600 
0012700 


1P10- P1NDE X{N) 

1 PH I = PJNOE XINU l-l 

RE AD( l UNI N, 501 0 Ml AL PHU ) , L = M U L 0 » M U H I 1 
P T AO ( 1 UNI N, 5050) I Al.PHMU (1. 1 , L -”010 . MU H I I 
IF (N.NE.LTRUE) 00 TO 100 
NR = NPARAM(N) 

WRI TE ( IUNOUT ,6360) 

READ( 1 UN IN, 5080) ( R F A IMU< L ) , L = 1 , NR 1 
WRTTE ( IUNOUT , 6355 ) < R E 4L MU ( L ) , L •= 1 , NR ) 

WRITE (IUNOUT, 6360) 

100 CONTINUE 

RE AO ( 1 UN I N , lNPFMT)(PREC<L),L=tPLO, IP HI) 

" WR! TF( IUNOUT ,60501 N, NP ARAM N ) , THET A IN ) 

WR t TE ( IUNOUT, 606 0 }< I AL PH( U , AL PH MS.) < L J ,L = «ULO,MUH I ) 

WR E TE I IUNOUT, 6090) 

CALL T R 1 ANG( PRFC I I PLOM NPAR AM(N > , H , PFMT, I ALPH(MULO) I 

CALL MTVFC( PREC ( [PL01 , AL PHMU ( MUL 0 ) , NP AR AM ( N ) , PMU ( KU L 0 1 I 
I F ( Lf VOUT.GE .81 WRI TE ( IUNOUT ,606 0 M I A LPH ( L I , PMU ( L I « l=MIILO, MU HI ) 
190 CONTINUE 


M = NPARAMI LT RUE 1 
JJ = MIJO [ LTRUE 1-1 
XMSE=0.0 
00 195 J - l • K 
JJ * JJ+l 

195 XMSF - XMSE * 1 ALPHMLK JJ1-REALMUI J))**2 
XMST - SORT ( XMSE*TAU t 
XMSF e SORT ( XMSE 1 
WRITE (IUNOUT, 6100) XMSE, XMST 
XMSE = 0.0 
XMST = 0.0 
00 197 1=1, MXT RTS 

197 N AVG ( H = 0 
PCS = 0.0 
CALL SANO(INITAL) 

IF(KPGfN) IN!TAI=NTLSV 

IF(JRSTRT) RFADUUNIN, 5090) INITAL 

IE( .NOT.RESTRT ) GO TO 200 

WRI TEI IUNOUT ,64251 

CALL BCREA0(NAVG(1) ,M50FAR 1 


200 CONTINUE 

WO | TE ( IUNOUT, 6800) IN1TAL 
6500 FORMAT { 26M STARTING VALUE FOR R A NO = 
CALL ACOMB 
CALL SVSTP.T 


MMM= j 

rF(PESTRT) MMM=MS0FAR*1 
OO 800 M= MMM, NS YM 


0121 
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0 0 1 7 8 00 
0017900 

ocnooo 

0013)00 
00 1 3?OC 
001 3300 
0013600 
001 3500 
0013600 
0013700 
0013800 
00139C0 

001 A 000 


0 0 1 6 1 00 


001AJOC 


0016300 

59 0 

0016600 

6 CO 

0016500 


0CL6 o 6C0 


0016700 


0016BO0 


0016900 

700 

0015000 

750 

0015100 


0015700 


0015300 


OC l 5600 


001 6500 


0015600 

800 

0015700 

C 

0015500 

C 

0C159C0 

c 

0016000 

810 

0016100 


OG 16700 


0016300 


0 0 1 66 CO 


0016500 


0C166C0 

ft 5 0 

0 016 7 O0 


0016800 


00 16.510 


O016870 

6600 

0 0 ! 6 9 0 0 

66 00 

OC 17000 


001 7100 


0017700 


0017300 


0017600 


001 7500 

1000 

001 7600 


0017700 


0017800 

c 

0017900 

f. * * $ * <■ 

0018000 

r 

0018100 

5010 

0018700 

5070 

0018300 

5060 

0018600 

505 0 


M$QFAR=M ■ 

CALL G1STRT 

yo 700 ntr-i,kxtrts 

S'OftS- NTR 
SIN F= 0.0 

Of) 600 mT=M,NPULL 
TRTM,NT= ITRT-1 

CALL M FORM • ’ 

CALL iNFMTr,' 

IF I X 1NF.IE.SINF 1 GO TO 600 
SINF = X1NF 

1 SV= T RT MNT ° 

DO 590 N=l,MHYP 
SV ( N1 = S I N 1 
RV( N) = R( N) 

CONTINUE 
CONTINUF 
T«TMNT= [ S V 

IF ( LFVOUT.GE . 3 > WRITFI I UNOUT ,6210) ISV.S1NF- 

CALL RNDM 

CALL Y G F N I 17 5 01 

CONTINUE 

CALI COUNT 

CALL T l M E 1 ( T NOW ) 

TPRNT = ( TN0W-TSTRT1 /3600. 

IFC TPRNT. LT.TMAX) GO TO ftOO 
IGOT0=? 

GO TO ft 10 
CONTINUE 


CONTINUE 

WR l TFI IUNOUT, 6260) TPRNT 
XX«S C =X'1SE/FL0AT (MSOFAR | 

X XM S T = XXM SF. * S OR 1 ( T AU 1 
ASN-0.0 

00 R50 1=1. MXTH T S 
ASN=ASN+FLOAT ( NAVGI I ) * | ) 

ASN= ASN/ riEJAT (MSOFARI 

WRITFI I UNOUT, 6 6 001 I N AVG I 1 1 , I = ] , MX! R T S 1 
WP I IF I I UNOUT , 6600 I ( I HCNT f I) , 1 = ) , NHYP 1 
FORMAT ( III 101 10) 

FORMAT ( LH 101 17 I 
PPCS = PC S /FLOAT f MSOr AR 1 

WRITFI I UNOUT ,63001 ASN.PPCS.XXMSE.XXMST.INITAl 
NIL SV=1NITAL 
KPGFN=. TRUE . 

GIJ TO (1,10001, 1G0TO 

CALL RCOUMiM NAVOf 1 ) .MSOFAR.O) 

WP t TE II UNOUT , 66 501 MSOr-AR 
STOP 




FORMAT I 1 31 6) 

FORMAT! 1 6, 3F 12.6. 1 Oil 1 
FORMAT I 1 3A6. A?| 

FORMAT ( 1 7F6.0) 
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00 18500 
0018600 
00 l H 7 00 
001 8 800 
0018000 
0010000 
00191 CO 
0019800 
0019600 
0019500 
0019600 
0019700 
0019800 
0019900 
0020000 
0020100 
0020200 
0020300 
0020400 
0020500 
0020600 
0020700 
0020800 


0000 l'OO 
00CC200 
0000300 
0000400 
0000500 
0000600 
0000700 
0000800 
00C0900 
00C1000 

ooni ioo 
0001200 
0C01300 
0001400 
0001500 
0CC1600 
0001700 
0CC1800 
0CC1900 
0002000 
0002100 
0002200 
0002300 
0002400 
0002500 
0002600 
OOC27CO 
0002300 
0002900 
0003000 


50 30 FORMAT! 6 16, 3F6. 0, 2L1 1 
5090 FORMAT (0 l 2 1 
C 

*«■ $ t> ** t- & «<■ <r <> $ * 6 fc <•# * * <■» F* <f<f *4 <• * 

6010 FORMAT 11 HI ♦ 1 3A6 , l A? 1 

6050 FORMAT! 1 1 H K F OR MODEL I3/16H NO. PARAMETERS J5/13H PRIOR PROD- G14. 
X^ ) 

6060 FORMAT! 51HKTHE PARAMETERS IN THE M UD EL AND THEIR MEANS A R f - - // 

X < ( 110,014.5 1)1 

6090 FORMAT l 44HKTHE PRECISION MATRIX OF THE PARAMETERS 1 S — 1 

6100 FORMAT! 4 9HK IN I T I AL DISTANCE OF PRIOR MEAN FROM TRUE VALUE 014.5/. 

X 18X, 16H01V10FD BY S l GMA , 1 5 X , G 1 4 . 5 1 
6210 FORMAT! LM 5G18.81 

6240 FORM A T ! 2 4H CURRENT EXECUTION TIME F10.31 

6300 FORMAT! 6HKASN= G14.2/6M PCS= G14.A/12H AVG.D1ST.= 014.6/ 

X 14H NORMALIZED = Gl4.fi/2H* 0121 
6355 FORMAT! 1 H fiG16.fi) 

6360 FORMAT! 1H 40(2H *1) 

6365 FORMAT ! 1 H 80I1H*I) c 

6425 FORMAT! 37H THIS IS A RESTART OF A PREVIOUS CASE) 

6450 FORMA T < 55H THIS CASE WAS TERMINATED BY CLOCK. DUMPING FOR RESTART/ 
X , 2 9H NO. SIMULATIONS COMPLETED = 16) 

END 


‘SUBROUTINE -AC OMR 

COMMON/UN! TS/ I UNI H, I UNUUT , M ASK , LFVOUT , »TMT (4 I 
COMMON/ A L PH/1 At_PH( LOGO > , 11 ALPH< 512 1 , ALPHMUI 1000 ) , R F AL mu ( 5 1 2 1 
X , NPARAM! 10) , MUCH 1 1 1 , PMU11 000) 

COMMON /CNTRL S/NHYP , NAI L »NF AC , NFULL , NFUIM1 , T RT MNT , NT R END, NOBS . L T R U E 


NALL = NPAR AM ( 1 ) -NT REND 
l FI NALL . LE . 0) GO TO 110 

00 10G K = l , NAl. L 

1 I Al PH t K I = IALPMIK) 
l 00 CONTI NOE 

110 lFINIIYP.LE.il RETURN 
C 

00 ROO N-2,NHYP 

M A X 1 = MUDIMM 1-1-NTRENO 
KK=0 

K=MU0(NI -1 
150 KK = KK * 1 
200 K=K+1 

IF(KK.GT.NALL> GO TO 420 
IF(K.GT.MAXl) GO TO 800 
2 50 IF! 1 ALPH< K) - I 1 AL PH (KK 1 1 300, 1 50, 400 
300 KS=NALl*2 

DO 350 ,J:-KK , NALL 
KS^KS-l 

350 I I ALPHt K SI = 1 1 ALPHI KS- U 

1 1 AL p H ( KK I = t AL P H { K 1 
N 4 L L = N A L L ♦ 1 
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DOOM 00 


f.n TO 150 

C003?00 

4 00 

K K = K K *■ 1 

occnco 


IE1KK-NALL) 250,250.450 

0003400 

420 

1 F f K.OT.MAXI 1 GO TO 8 00 

QPC3500 

4 50 

NMORF=M AXl -K+l 

00C3600 


KD=K-1 

OC C37 0C 


00 500 J = 1 , MM ORE 

0 003800 


«.St = NALL *J 

00C30CC 


KS2-K0+J 

0004000 


I [ ALPH( K S 1 J = I A L P H l K 5 ? ) 

0 C O'* 1 00 

500 

CONT i nue 

0 0C'*2 00 


NALL =NALl + NMC.IRE 

00 O'* 300 

ftOO 

f.ONT l NUE 

00C4400 


l FI LF VOMT .GE .7 lKRITFC IIJNOUT , 1000 1 NALL, III ALP HI 1 ) , 1*1, NALL 1 

OOC45CO 

l 000 

FORMAT { 1H [ 10/1 1H 25151 1 

0 0 O'. 6 00 


RETURN 0 

0C04 7 00 


END 


00001 00 


SUBROUTINE SVSTRT 

0000200 


COMMON/ AL PM / I At. PM ( 1 000) ,1 1 AL PM( 5 12 ) , ALPHMU ( 1 000 ) , RE AL MU ( 5 1 2 ) 

0000300 


X .NPARAMUO) ,MUO( 11), PMU( 1000) 

0000400 


COMKON/PS/PI NDEX 11 1) , PRECI2000 1 

0 0 00 5 CO 


INTFGER PINOEX 

0 0C06 00 


COMMON/C NT 9 L S/N HYP, NALL ,NFAC . NFULL .NFULM1 , TRTMNT , NT RENO, NOBS, LTRUF 

0 0007 00 


COMMON /PRORS/THF TA ( 10) ,CSTOP 

00 OCR 00 


01 WENS 1 ON XL PH Ml J( 10 00 ) , XPREC ( 2000 » , X THE 1 l 1 0 ) . XP MU 1 1000 1 

oocoooo 


M= MUO (NHYP* 1) - 1 

00C1000 


no 20 J=1 , M 

OOCt 1 00 


XLPMMU(J) = 4L PHMU t J 1 

OCCL200 


XPMUt J) - P WU ( J 1 

0C01300 

20 

CONTINUE 

0001400 


M = PINDEX(NHYPtlJ-l 

00C1 500 


00 40 J = 1 , M 

0001600 


XPRECIJ) * PREC(J) 

0001 700 

40 

CONT [NUt 

0OC1 800 


DO 60 J= 1 , NHY P 

00 Cl 900 


XTMETIJ) * THETAIJ1 

0002000 

60 

CONTINUE 

0002100 


RETURN 0 

0CC22 00 


ENTRY GTSTRT 

0002300 


M- MUDINHVP+ 1>-1 

00C2400 


DO 120 J=1 , M 

000? 500 


A L PHMU I J 1 = XL PHMU I J ) 

0002600 


PMU( J) = XRMiM J ) 

00C2700 

1 20 

CONTINUE 

0 0 C 2 8 0 0 


M = P INDE XI NMYPf t » - 1 

0002900 


DO 140 J=l,M 

00C3000 


PREC(J)= XPREC(J) 

0003100 

140 

CONT INUF 

0003200 


DO 160 J r 1 » N HY P 

0003300 


THETA! J) = X THE. T ( J ) 

0003400 

160 

CONTINUE 

0003500 


RETURN 

00C3600 


END 
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0000100 
0C001 50 
0000200 
0000300 

oocoaoo 
0000500 
0000600 
0000700 
0OCOROO 
00CO900 
0001000 
00C1 100 
0001 200 
0001300 
0001400 
0001500 
0001600 
0001 7 00 
0001 800 
oocioco 
0002000 
0002100 
0002200 
00C2300 
00C2400 
0002500 
00C2600 
0002700 
0002800 
0OC2900 
000-3000 
0003100 
0003200 
0CC3300 
OC 03600 
0003 500 
0 0036 00 
0C037C0 
0003800 
0CC3900 
0004000 
00 CM 00 
0004200 
0004300 
O0C440O 
00C45C0 
0 0 04 6 CO 
00C470D 
0 004 800 
OCC49C0 
00C5000 
0006100 
0005200 
00C6300 
0005400 
OOC68CO 
00C6600 
0CC5700 
0006800 


C THIS 15 FOR TWO- 1 E VEL FACTORIALS ONLY 
SUBROUTINE M FORM 

COMMON /UN 1 TS/ 1 UN 1 N , 1UN0UT t MAS* , IFVOUT , PF- MT (41 
COMMON/ALPM/ | ALPHI 1000 > , 1 I AL PH I 5 1 2 ) , ALPHMUUOOO 1 , R C AL MU < 5 l 2 1 
X , NPARAMt 101 . MUD ( 111, PMUt 3000) 

INTEGER TRTMNT 

COMMON/CNTRLS/NHYP, NALL ,NF AC . NFtlLL , NFULMJ , TRTMNT ,NTRFN0,NO8S,LTRUF 
COMMON/MM/OtSM( 1000) 


C. 

* $ * * ** * * * # tf * * * * * * * * * <■* ♦ * ** ** * * ** * * 

C 


01 KFNS10N LASTAUO) 
EQUIVALENCE (KS.SKI ,( IX.Xll 


DO 5 N= l , NHYP 
LASTA(N)=0 
5 CONTINUE 


DO 1000 1=1, NALL 

[PARAM = I1AIPHIII 
IF( IP4RAM.NE.01 GO TO 40 
A= +1.0 
CO TO 800 


40 CONTINUE 
itr=trtmnt 
r>x= i.o 

on 150 J=1,NFAC 
XI = ANO (MASK , {PARAM) 

■rx= i x+ 1 

GO TO I 1 30, 100 1 , IX 
100 SK = ANO< MASK, UR 1 
KS= KS+1 

GO TO ( 1 10, 1 30) ♦ KS 

no dx= -ox 

130 !PARAM= 1PARAM/2 
1 TR - I1R/2 
180 CONTINUE 
A = OX .. 

C 

c 

500 CONTINUF 

IF (LEVOUT.GE .7 ) WR l T E ( I UNOUT , 6000 ) IIALPHIIJ.A 

c 

00 950 KM » NHYP 

1F( LASTAIK l-NPARAMI K l ■» N I R FNO ) 520,950,950 

520 I X= MUOI K) +L ASTA I K ) 

IF ( I ! AL PHM 1 - 1 A L 0 H ( IX) 1 950,550,9999 

550 CONTINUE 

L A S T A ( x ) = LASTMKJ +1 
06SMI 1X1= A 
950 CONTINUE 
1000 CONTINUE 

IF( NTRENO.LE .0 ) GO TO 1060 
A = NO«S 
P= l.OEO 

O0 1050 J= l , NT RENO 

e= b+a 
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occ59co 

OOC6000 

ooo6ion 
00062 00 
0006100 
00C64D0 
0006500 
OCC6600 
0006700 
0006800 
0006900 
0007000 
00C71 00 
00 C7200 
00 07 100 
0007*00 
00C=7 5 00 


0000100 
0000150 
0000200 
0000100 
0000400 
0000500 
0000600 
0000700 
0000000 
oncoooo 
0001000 
0001 l 00 
00 012 00 
0001 100 
0001400 
0001 500 
0001600 
00C1650 
0001 700 
00C1800 
0001000 
0002000 
0002100 
0002200 
0002300 
0002400 
0002500 
0002600 
0002700 
0002800 
00C290O 
0003000 
0003100 
0003200 
0003300 
0 C 034 00 



K S= J-NTRF 

: NO 

. 


nn 1040 * 

;=i 

, NHYP 


ks=ks+nparam 

no 


desmiks )= 

B 


1040 

CONTINUE 



1050 

FONT I NUE 



1060 

C FIN 1 1NUE 




I TR=MUD( NHYP 

H 1 - 1 


IF 1LEV0UT.G 

E.7) WRITE ! 1UN0UT ,6010] < DF SM III , 1 = l , 1 T R I 


RETURN 


£• 

9999 

CALL F X I T 




RETURN 



6000 

FORMAT ( 1H 

1 6 

, F 6 . 0 ) 

6010 

FORMAT! 6H 

0 F 

SM= /!6X,15F3.0U 


FNO 




C THfS 16 FOR POLYNOMIALS ONLY 
SUBROUTINE MFORM 

COM MON/ UNI T$ 6 HJNIN, I UNOUT , MASK , L FVOUT, Pf MT (4 I 
COMMON/ A L PH/ I AL PH! 1 OOO ) , U AL PH l 5 l 2 I , ALPHMU ! 1 000 ) , RC-AL Mil ( 5? 2 1 
X ,NPARAM(101 .KUO( 1 1) ,PMU( 1 000) 

INTEGER TRTMNT 

COMMON /C N TR LS/NHVp., NAL L , NF AC ,NFUL L , NFUL M 1,, T RT MNT , NT P. END, NOBS, L TRUE 
C0 M MON /MM/OF 5 M ( 1000 ) 


C = 

£ * $ f f ft ft * * ft ft * t f * 1 1 ft ft Ft ft ft 

c 

D1MFNSI0N LAST A { 10) - 

EQUIVALENCE ! KS , SK ) , < I X, XII 

C 


C 


C 

C 

c 


C/X=?.0/FLOAT(NFULL-1 i 
X=- l .O + FLOAT (TRIM NT I * 0 X 
IF< ABS( X ) ,LF. 1 .OE-4 1 X = 0.0 

00 5 N=1.NMYP 
l AST A(N)=0 

5 C PINT 1 NUE 

DO 1000 1=1, NALL 

1 PAP AM = I I ALPH! [ I 
A = X * * 1 P ARAM 


500 CONTINUE 

IF( IEVOUT.GE .7 1 WRITE! I UNOU T , 6000 ) 1IAI.PH<]I,A 
DO 950 K= 1 , NHYP 

IF! LASTA!K) -NPAPAM(K) f NTP END ) 520,950,950 

520 I X= MUD (K)+LASTA(K1 

1FIIIALPIUI) - I ALPH! T X » ) 950,550,9999 
550 CONTINUE 

L A S T A ( K I = LASTA! K) ♦ l 
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0003500 

00C36C0 

0003700 

0001800 

0003900 

oocaooo 

0 0 04 i 00 
ooo4?oo 
000430C 
000^400 
000A500 
0004600 
00C4700 
0 004800 
0004900 
0000000 
CC051 00 
0008/? 00 
0C05300 
0005400 
0005500 
0005600 
0005700 
000*800 
O0C5900 


OOCOIOO 
0000200 
0000300 
00C0400 
00C0500 
0000600 
0000700 
0000800 
0CC0900 
0001000 
0001 100 
00C120C 
0001100 
0001400 
0001 500 
0001600 
0001700 
OOC] 800 
00C19C0 
0002000 
OOC?. 1 00 
0002200 
0002300 
0CC2400 
0 0 C 2 5 0 0 
C 0 C?6 00 
000? 7 00 
0002800 


DESK! IX )= A 
<3 50 CONTINUE 
1000 CONTINUE 

IF { NT KfND. LE .0 1 00 TO 1060 
A = NOBS 
B= UOEO 

DO 1050 J= 1 , NTREND 
B - U*A 

K S= J-NTRENO 
DO 1040 K= 1 . NHYP 
K S=KS*NPARAM(K) 

OF. SM [ KS I = 8 
1040 CONTINUE 
1050 CONTINUE 
1060 CONTINUE 

1TR=MU0( NHYP+t 1-1 , TD . 

IF ( Lt VOUT . GE . 7 1 WR IT F ( 1 UNOUT . 60 10 ) ( OE$M ( 1 1 t ! - 1 » 1 T R 1 

return 

9999 CALL EXIT => 

RETURN 

6000 FORMAT (1H I 6, F6.0I 

6010 FORMA T ( 6H OESM = / ( fcX , l 5F 8 . 0 ) > 

END 


SUBROUTINE INFMTN 

COMMQN/UN ITS/ I UNI N, I UNOUT .MASK. LF VQUT . PFMT (A 1 

COMKON/C NTRLS/ NHY P ( N ALL . NF AC i HFULL i NFUL Ml i TRTMNT , NT RE NO f NOB 5 t LT RUE 

INTEGER TftTMNT 

COMMON/ XRAND/Y, EPS. UN IF .U 

COMMON /E RR / T AU » SIGMA 

COMMON/ ALPH/ IALPHI 10001 . II ALPH151?! T AL PHMU 1 1000 1 . RE At MU ( 5 1 2 1 
X « NP AR AM (101 ,MUO( 11 l.PMUUOOO) 

COMMON/ PS /PI NDF X (111. PR EC (2000 1 
I N T E G F ft P INDEX 
COMMON /INf/XINF 
COMMON/ PROPS /THE TAUO 1 .CSTOP 

COMMON/MM/DE SM( 1 000* c 

COMMON /YD r ST/ S(101,R(10I.SV(10>,RV(10> 

CQMMON/X TRA/ AA ( 1000) , FM2000) ,C(2000»,n( 10) 

DIMENSION DFSMUUO) ,TRACE( 10) 

LOGICAL UP DT 


C ^ ^ ********************** 

C 


*********************** 


GO TO 5 
ENTRY UPDATE 


UPOT = . TRUE . 


GO TO 6 

5 UPOT=. FALSE. 

6 CONTINUE 

DO 10 K= 1 i NH Y P 
0ESMU(K1=0.0 
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00C2900 
Q0C3000 
00031 00 

oocn?oo 

0CC33O0 
0OC3400 
OOC3SOO 
0003600 
0003700 
0(5 C3800 
OOC3900 
0004000 
00C41 00 
0 0047 00 
0004300 
0 004 4 00 
0004500 
00 04600 
0004700 
0004800 
0004900 
0008000 
00C51 00 
0008200 
0008300 
0006400 
00 CSS 00 
0005600 
0005700 
00 058 00 
00C5900 
0006000 
0CC61C0 
0006200 
0006300 
00C6400 
0006500 
0 0066 00 
0006700 
0006800 
00C6900 
0007000 
0007100 
0007200 
00C7300 
0007400 
0007500 
0007600 
0007700 
0007800 
OOC7900 
OCOROOO 
0 0 08100 
0008200 
0008300 
0 0 084 00 
0008500 
0 0086 00 
00C8700 


10 CONTINUE 
C 

k^^*^***,*^**,#*** «.***«.**** <■-<<*#*** **■***<! *$***$<: tifr#**** 

C 

DO 520 N=1 iNHYP 

IT- (LEvriUT. CE .5 ) WR ITE ( IUNOUT, 60051 N.N08S 
KS= MUOINl 
JE = N p A R A M ( N 1 
C 
C 

10=0 

00 ioo j=i,je 

• KS1=M$+J-1 

UIUPOTl GO TO 25 

DESMIJIN )=0E SMUIN l + DESMI KS 1 ) * At PHMU ( K S l ) 

25 CONTINUE 0 

. DO 100 JJ=1»J 
TB= 18+1 
KS? = MSfJJ-1 

B I 1 R 1 - DESM( KS1 )*0ESMIKS2) 

100 CONTINUE 

C ***************** re********************** 

C 

I F ( l.F VOUT . GE . 6 1 CALL TR I ANG ( B, JE. ft, PFMT , IALPH( MS 11 

KSl=PINDEX(N4-t 1-PlNOEXINl 

K S 2 = P I NDE X 1 N 1 

DO 120 K- 1 • KS 1 

B(Kl = TAU*R( K1 *-PRFC (KS?1 

C ( K 1-=R ( K ) 

KS?= K5?+l 
120 CONTINUE 

CALL INVXTXCB, JE1 

irUEV0UT.GE.6l CALL TR I ANG ( B t JE ♦ 8 , P FMT , I ALPH ( MS 1) 

C 

Qt* $ $ * Xftr ♦ * If * * * * * <t * * $ ♦ * - * ♦ * * * * * ^ * * 

1 F I . NOT ♦ L1P0T 1 GO TO 199 
KS1=P1NDEX (N+l 1 

IPLO = P i NDE X < N 1 
KSUKS1-IPL0 
MUM J=MU01N+1 1-1 
KS2= I PLO 
00 130 J=1,K$1 
PRECUS2 )=C ( J 1 3 

KS2=KS2+1 
130 CONTINUE 
KS1 *MS-1 
00 150 J= 1 ♦ JE 
K S 1 = KSl + l 

AA{J1= Y*0ESM(KS11*TAU + PMUIKSIJ 
150 CONTINUE 

CALL MTVECIB.AA, JC-, ALPHMUIWS1 1 

CALI MTVECI PREC( l PL 0 1 •» AL PHMU (MS I » JE » PHU{ MS 1 1 

IFILEVOUT.LT. 3) GO TO 520 

IF( LEVOUT.GF .5 1 GO TO 160 

WRITE! IUNOUT , 6020 K ALPHKU ( L 1 1 L = MS ,MUH I 1 

1F( LEV0UT.LT.4l GO TO 520 

WRI TE < IUNOUT .60901 c 

CALL TR [ ANG(PREC{ IPL01 , JE .B.PFMT, IALPHIMS IV 
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00C8R00 

00C8900 

occqooo 
ooc^i oo 

0009200 
00C9 300 
0GC9900 
00G9500 
O0C9600 
00C9700 
00C9800 
0OC9900 
0010000 
0CI01 00 
0010200 
0010300 
00 109 CO 
0010500 
00106C0 
0010700 
0010800 
0010900 
0011000 
0011100 
0011200 
0011300 
0011900 
0011 500 
0011600 
00-1 17 00 

oonaoo 

0011900 
0012000 
0012100 
0012200 
0012300 
0012900 
0012500 
0012600 
00)2700 
00 1 2 R 00 
0012900 
0013000 
0013100 
0013200 
0013300 
0013900 
0013500 
0013600 
0C137O0 
0013800 
OC 1 3900 
0019000 
0019100 
0019200 
0019300 
0019900 
0019500 
0019600 


GO TO 520 

160 continue: 

WRITE! 1 UNO 111 ,6060 M I Al PH! L ) , AL PUMU ( l > »PMU(L ) , L = MS ,HUHl ) 
WRITE! IUNOUT, 6090) 

CALI TR I ANG < PR EC ( IPL0),JF t 8,PFMT,]ALPHtMSM 
GO TO 520 


C 

C 

C. 

C 


199 CALL MTVEC!B,DFSM(MS),JE.AA) 

DO 300 KS2=1,JE 
°2 S 0 A A 1 K S? ) = T AU* AA ( KS21 
300 CONTINUE 

[F ( LEVOUT .GE .5 ) WRITE! 1UN0UT . 601 0 l < A A{ J > , J= 1 , J E 1 
C a 

£ A A . A A $ $ * if f A A A A* A A C A A * * A A A A A A A A A A A A A A 

C 

R { N ) = 0,0 

*MS=MS-1 

DO 360 J= 1 » J£ 

W.HS= MMS+i 

P ( N 1 = R«N) + AA( J)*OESH(MMS) 

360 CONT l NUF 

R ( N ) = TAU* ! 1 .O-R(N) I 
C 

E A A A a a a a a a a a a a a a a a a * * a A A A A A *9 A * a A A A * A 

C 

MMS-MS 

.C=0.0 

00 500 J= I , JE 
C e C + A A ( J ) * PMU! MMS ) 

MMS= MMS *-1 
500 CONTINUE 

S(Nt=C/RtN) 

IF ! LEVE1UT.GE .5 > WR I T E < l UNCIUT , 6020 ) R(N),S(N) 

520 CONTINUE 
C 

IF(UPDT) RETURN 

£ A A A 6 A A A A A A A A A A A A A A A A A A A A A A A* A A A* A A AA 

r. 

no 1000 N= 1 , NHYP 
C = 0.0 

no 5H0 J = 1 t NHYP 3 

IFfN.EO.J) GO TO 580 
C*C*THETAfJ) *P(J) 

580 CONTINUE 

TRACE ! N 1 =C 7R <N ) 

IF! LFVUUT.GE .5 ) WRITE! IUNOUT ,6035) TRACE ( N ) 

1000 CONTINUE 


C 


XINF = THETA(l) a TRACE!)) 

DO 1500 N=2 , NHYP 

JE= N— 1 

XINF = XINF ♦ TMETMN) A TRACE < N > 
DO 19 50 J = 1 , J E 
C= S ( N 1 - S < J ) 
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001^700 
00 1 4000 


0 = 0*0* ( R l N1 +R< J n 
XI NF = X | NO 4- THETA (N I *THCl"«‘ u 


0014000 
0014000 
0014100 
00 1 5? 00 
0014300 
0014400 
00 1 4400 
0014600 
0014700 
00 14800 
001 5^00 
0016000 
0016100 
0016300 
0016300 
0016400 
0016500 
0016600 
0016700 


1440 CONTINUE 

1500 CONTINUE tn-»oi v;‘,> 

IF ( LEvnuT.GE. 5 J WRI TE M UNOlU » 

RETURN 

,**«.« ♦******«<( * 


£ * * # * 4 t < »#$«■**' 

C FORMATS 

6005 FORMAT (3H N= 16, 7H NORS= 16 I 
601 0 FORMAT! 2H A/(1H 10G12.AU 
60? 0 FF1RMAT ( 1H 10G12.4) 

6035 FORMAT! 7H TRACE- G 1 2 » A 1 K r j p r { 

6060 FPRHAT(51HK THE PARAMETERS IN 
X < < 1 H 110,2014.5111 
6090 FORMAT C 44HKTHE PRECISION MA 1UI' 

END 


* * <' *****fe*«s<»**********fc**i(<* 


* * <■ $«.-**fc****m«*<i*:***+****<t£ 


AVI THEIR MEANS ARE-- // 
RtRiV.ETEKS IS — ) 


0000100 
OOC0200 
0OCO3OO 
OU00400 
0000500 
0000600 
0000700 
OCCOSOO 
OCC0900 
OOC) oco 
0001100 
0001 200 
0001300 
0001400 
0001 500 
0001600 
0001 700 
OOQlftOO 
00C1 900 
00C2000 
0002100 
00C2200 
00C2300 
0002600 


SUBROUTINE RNOM 

COMMON/ XRANO/ Y-, EPS , UN I F , U 1FV0UI 

COMMON/UNITS/ IUN1N,MJN0UT,MA U- ,LEVUUI- 
COMMON/ERR /T AU. S 1 GMA , 

DATA AO/2. 515617/. Al/.B02H4y.J?'-OiO-.^/ 
X Bl/ 1.4327R8/,B2/.lH9269/«>^ /<001 '?'!»/> 

X IIHIN/. 0000001/ 


U = U 

CALL RANDtUN IF 1 
U=UN1 F 

l F ( UN 1F.GT..50) U=1.0~UN|F 
I F ( U« L T , UM1 N ) GO TO 100 
T2 = ALOG 1 l .0/<U*U) * 


T — SORT U2 I iiH 1 1 R?^ * 

FPS = T-< A0*Al*T+A2*T21/1l.0‘t' *;*”< f . V ‘ 
IF (LFV0UT.GE.7 1 WR 1 T E ( ItlNUH I , 6 000 1 IW\ V 
IF(UN1F.LT. .501 EPS=-EPS 


cps=eps*sigma 


RETURN 

100 F p S=- 1 . OE 1 5 

I F ( UN IF.GT..50) E p S=-EPS 
RETURN 

6000 FORM A T ( 6H RANO /1H 6G16* 1 '! 
FND 




~*T2 1 

_ T,T2,FPS 


c 



100 


OOODIGO 
OP GO? 00 
0000300 
0 0 00 A 00 
0000500 
0000600 
0000700 
0000800 
0000000 
000] 000 
0001 1 00 
0001600 
0001600 
0001700 
0001800 
00C1900 
0007.000 
00021 00 
0002200 
0002300 
0002600 
0002500 
0002600 
0C02700 
0002800 
00C2900 
0003000 
00C3100 
0003200 
0003300 
0003600 
0003500 
0003600 
0003700 
0003800 
0003900 
0006000 
0 006 1 CO 
00C4200 
0006300 
000^00 
0006100 
0006200 
0006300 
00C66 00 
0 006 5 CO 
0006600 
0006700 
0G06RO0 
00C4900 
0007000 
0007100 
0007200 
0007300 
00 07400 
0007500 
0007600 


SUBROUTINE YGEN!M 

COMMON/UNI TS/IUNIN, IUNOUT , MASK, I. EVOUT « f'FMT (4 1 

COMMON/At PH/1 41 PU( 1 0001 , If At PH( 5 12 1 , ALPHMIJ UOOO) , RE AL MU 15 1 2 ] 

X , NP ARAM (101 , MUDt 1 11 ,P8U( 10001 

COMMON /C NT Rt. S/NHYP, NALL , NF AC , NFULLt NRJLM1 , TRT MNT , NTR END , N08 S » L TR UF 
INTEGER TRTMNT 
COM MON/ MM / 1JE SM ( 1000 1 
COMMON /XR AND/ Y , F PS ,UN ! F ,U 
COMMON/ YO 1 ST/S< 1 0 ) , R l 10 > . $ V ( 10 ) , RV I l 0) 

COMMON/ PR08S/T HFT All 0 1 , CSTOP 

CCIMMON/PRF MNC/NAVG( 10 00) ,PC$.XMSF,XMST,1NITAL , IHCNT!10),MSOFAR 
EOUI VALENCE ( SK ,KS 1 , M X, Xl> 

C 

C 

CALL MFORM 
Y^EPS 

M = NPARAMI L TRUE 1 
JJ = MUD ( L T RUE. 1 - 1 
DO 550 J=1 i M 
JJ = JJ *1 

Y = Y * DESM< JJ l*REALMU{ J> 

550 CONTINUE 
SUM = 0.0 

00 700 N= 1 , NHYP 
C = Y-SV<N) 

Q = K V t N 1 *C c 

S ( N I = S V ( N ) 

R(N) =RV< N) 

0=-.50*0 

1 F( ABS! Q) -60 .0 > 660,670,670 

660 IF! THE YA ( N ) - 1 . OF- 1 8 I 670,670,680 c- 

670 ThETAINI = 0.0 
GO Tn 700 

680 A = ALOG ( THE T A { N ) 1 ♦ Q 

IF ( ABSI A ) . GE .70 .0 ) GOTO 670 
690 THETA1N) = THETACNI * SQRTIRIN)! * EXPCO) 

SUM = SUM <• THETA < N ) 

700 CONTINUE 
1=0 

00 710 N= 1 , NHYP 
710 THETA(N1 =THET AIN) /SUM 
DO 920 II =1 , NHYP 
1FITHETAI I l)-CSTO p ) 920,910,910 
910 f=l 

GO TO 1000 
920 CONTINUE 
1000 CONTINUE 

1F(LFV1UT.GF:.2> WRITE! IUN0UT.6010) TRTMNT, ( TH CT A I N 1 , N= 1 , NHYP ) 

6010 FORMAT! 1 H I10.10F10.4) 

IF < LFVOUT.GF .5 I WRITE! IUNOUT.6000) Y , £P $ , ($(N),R(N), THE TAIN) , C ! N I , 
X N = 1, NHYP) 

6000 FORMAT! 3H Y=G12.5,5h EPS=G12.5, l 2H S,R, THETA, C / 

X I 2 X , 4G 16.6) ) 

CALL UPDATE 

IF! I .EQ. 1 1 RETURN 1 

RETURN - 

FNO 
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0000100 
CCC02C0 
00003C0 
0000400 
0000500 
0000600 
0001 000 
0001100 
0001 300 
00 014 00 
0001500 
0001600 
0001700 
00 01 000 
0001000 
ooc?ooo 
0002100 
0002200 
0002300 
0002400 
0002500 
00C4200 
0004300 
0 0 04 4 00 
0 004 5 00 
0 0 04600 
0 004 7 DO 
0004800 
00C4900 
0004 <550 
0005000 
0005100 
00C5200 
0005300 
0005400 
DO 055 CO 
0005600 
0005700 
0005800 
0005600 
0006000 


0000100 

0000200 

00C0300 

0CC04C0 

0000500 

00C0700 

00 COR 00 

0001 5C0 
0CC1 600 
0OC1 7 CO 
OOCI^CO 
0CC2000 
0OC2J 00 


SUBROUTINE COUNT 

C0MM0N/UN1 T$/ 1 1JNI N, lUNOUT , MASK, LEVOUT 

COMMON/ A L PH/ I ALPHI 1 OOC) , 11 AL PH( 512 ) , ALPHMUI 1000) , R E Al. M ! J I 5121 
X ,NPARAM<!0) , MODI l 1) ,PMU( 1000) 

COMMON /C NT RES/ NHYP, NALL « NF AC, NFUU. , N FUL Ml T T RT MNT , NT RE NO, NOBS , L T R U E 
C0MMON/PR03S/THETA< 10) ,CSTOP 

COMMON/ P RE MNC/NAVGl 10001 ,PCS,XMSE,XMST,INITAL , I HC N T(l 0 ) , M SOF A R 
COMMON/ erp./tau, SIOMA 
XM = 0.0 

NAVG1N08S) = NAVG( NOBS 1 *1 
M = NPAR A R( LTRUE ) ° 

JJ = MUIH LTRUE 1-1 
00 5 J=1,M 

JJ = JJ +1 

XM = XMt( ALPHMU< JJ) -RE ALMU( J )) **2 
5 CONTINUE 

XMSE = SORT ( XM ) X MS F 

XM$T c SOP- T < XM* T AU ) 

XM = SORT(XM) 

IE <LEV0UT.GT.1J WRITE) IUNOUT, 6000) NOBS , X M , X.MS T 
1MIN = 0 
C MAX-0.0 

00 910 1=1, NHV P 

IF ( THET A( Il-CMAX ) 910,910, 905 : 

905 C MA X = THE T A ( I ) 

1 MI N = I 

910 CONTI NUF 

945 CONTINUE ' 

IE < JMJN.EO. LTRUE) PCS = PCS* 1.0 
!HCNT< JM IN)= IHCNT< IM[N ) +1 
IE < LEVOUT.LT. ? ) RETURN 

00 950 N~ 1 , NH X P c , 

MULO = MUO<N) 

MUH) = MUD< N+ 1 >-l 

WRITE < IUNOUT ,6010 ) THETA(N) , t ALPHMU ( U , L =MULO, MUH ! ) 

950 CONTINUE r 

RETURN 

6000 FORMAT (22 H ** + * **0B SE R V A T t ON NO. 14.7H XM$E = GU.3,7H XMST = Gil. 3 
X ,6H* ****<< ) 

6010 FORMAT ( IH F10.6 , 10012.4/(27X10012.4)) 

END 


SUBROUTINE 1NVXTX(A,NN) 

C ASSUMES THF MATRIX A IS SYMMETRIC AN 0 POSITIVE DEFINITE, AND ONLY 

C THE UPPER TRIANGLE IS STORED AS A ONf-O IM£N$ IONAL ARRAY IN THE 

C ORDER A ( 1 , 1 1 , A ( 1,2), A<2,2>, Ad. 3), AI2.3), A(3,3), ..., A ( N , N ) . 

C NN IS THE ORDER N OF THE INPUT MATRIX A. 

C 

DIMENSION Aid 
0 = 1 .0 
K- = NN 
I TR I = 0 
DO 145 K= 1 ,N 
C 

1 TR 1 = 


ITRl+K-1 
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00022G0 KPl » Kfl 

0002300 KM1 = K-l 

0 0C2400 KK = JTK14K 

00026 00 PV = l.OOO/^tKK) • 

0002700 C 

0 0 C2 ROO I TP 2 - 0 

00C29OO IF (K-l 1 1 5 0 i 50, 50 

00C3Q00 C 

00C3100 C REDUCE TOP PART OF T R l ANGLE ♦ L F FT OF PIVOTAL COLUMN 

0003200 50 00 60 .1=1, KM 1 

0003300 1 TR? = 1TR20-1 

0 0034 00 o K J * I T R 1. J 

0003500 ° F = AIKJHPV 

0003600 DO 60 i = 1 * J 

0003700 1J = ITR2+1 

0003800 IK = ITP1 4 I ° 

00C3RC0 60 A ( I J I = A < I J » + A(lKl*F 

0004000 c 

0004100 IF (K-NI 70,120,150 

00C4200 C 

0004300 C RFOUCF REST OF TRIANGLE, RIGHT OF PIVOTAL COLUMN 

0004400 70 ITR2 = I TP t 

0004500 RO DO 110 J=KP 1 , N 

0004600 I TP 3 = 1 TR 1 

0004700 I TR 2 = ITR2 + J-1 

OOC4ROO KJ = ITR2+K 

0004900 F = A(KJ)*PV 

0005000 00 100 £ = 1 , J 

0005100 IF (I-KI 90,100,95 

00052 00 90 1J = ITR24-I 

00C5300 IK = I TR 1 * I - 

0005400 A ( I J ) = A I I J I - A ( l K I * F 

0005500 GO TO 100 

0005600 95 FJ = I TR2 +1 

0005700 J TR 3 * ITR3 4-1-1 

00C5R00 IK = ITR3 4 K 

0CC5900 A ( l J t = A ( I J I - A(lKi*F 

0006000 100 CONTINUE 

0006100 HO CONTINUE 

0006200 C 

0006300 C OlVIOE PIVOTAL ROW-COLUMN BY PIVOT, INCLUDING APPROPRIATE SIGNS 

00064 CO 120 I TP 2 = ITftl 

0006500 00 140 1=1, N 

0006600 IF * I -K » 125,130, 1 35 

0006700 125 IK - 1TR141 

OCC6ROO A ( I K 1 = -A(1K1*PV 

0006900 GO TO 140 

0007000 C (REPLACE PIVOT BY RECIPROCAL! 

0007100 130 AIKKJ ■= PV 

0007200 GO TO 140 

0007300 135 1 TR2 = ITR2+1-1 

0007400 K 1 = I TR2 + K 

00C7500 A ( K I 1 = A I K I 1 * PV 

0007600 140 CONTINUE 

0007700 C 

0007R00 145 CONTINUE 

00C7900 C 

0 OCR 000 150 RETURN ' 

OOCftlOO FNO 
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0 0001 00 
0QC02 PO 

oocoooo 

0000400 1 

0 0C05 00 
00C0400 
0000700 
0000800 f. 4 
oocoroo 
0001000 
0 CGI 1 00 
0001200 
00C1300 
0001400 c 
0001500 

oociooo 

0001700 
0001800 ' 
oociooo 
0002000 
00C21 00 
0 0.C22 no 

0002300 
0002400 30 

0002500 
00C2O00 
0002700 
0002800 
OCC2900 
00C3000 60 

C0031 00 
000.3200 
0 003300 
0003400 90 

00C3500 C 
0 0 03600 120 

0003700 
OOC3ROO 
OOC3900 
0004000 
OOCM 00 
0004200 
0004300 
0004400 
00C4500 150 

0004600 C 
0004700 180 

0004800 


SU8R0UT 1 NE TR 1 ANGt A , NN, NKOL t FORMAT , I OOUT I 
P 1 Mf NS I ON Mll.F 0 R M A T ( 11 , I OOUT ( 1 1 
COMMON/ UNI TS/ IUNIN, IUNOUT 
FORMAT ( 1 UK) 

N = NN 
NCOL = NKOL 
KLOMPS - N/NCOL 

KEEPTR = 0 
K 1 =1 

K2 * NCOL - 1 

K3 = NCOL ° 

IF (KLUMPS . E 0 » 01 GO TO 120 

DO 90 KLUKP=1,KLUMP5 
1 TR 1 - KFEPTR 

I = -l 

ILO = 1 KLUMP-l l*NCni + I TR 1 + 1 

00 30 K=Kl , K2 
1=1 + 1 

1 TR1 = I TR1 + K - 1 

ILO =11. 0 + K — 1 

I H 1 = ILO + i 

WRITE! IUNOUT .FORMAT > IDOUT (Kl,(A<Jl.J=ILO* I H I > 
KEEPTR = I T R 1 + K2 

00 60 K = K 3 , N 

1 TR 1 = ITP.l + K - l 
ILO = ILO + K - 1 

I H 1 = ILO + NCOL - 1 

WRITE! IUNOUT .FORMAT II D0UTIK1 » ( A < J 1 , J = I L □ , 1 H 1) 
K1 = K1 4 NCOL 
K? = K2 + NCOL 
K3 = K3 ♦ NCOL 
WRITE! IUNOUT, 1 I 

t TR 1 = KEEPTR 

IF ! K J » GT . N) GO TO 180 

I = -1 

ILO = KLUMPS+NCGL + IJPl + 1 
DO 1 50 K = K 1 , N 
1 = 1 + 1 

I TR 1 = 1 TR 1 + K - 1 
ILO = ILO + K — 1 
I H I = 11.0 ♦ f 

WRITE! IUNOUT , FORMAT 1 I OOUT (Kl , 1 A! J1 ,J=ILO,lHI ) 

pfturn 

END 
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0000100 

oooo?oo 

0000300 

0000400 
0 00.05 00 
0000500 
0000700 
0000500 
00C0900 
0001000 
00C1 1 00 
00 Cl 200 
0001 300 
OCCJ^OO 
0001^00 
0001500 
0001 700 
0001000 
00C19C0 


SUBROUTINE htvecu , h,nn,ci 
DIMENSION Ml 1,011 ),C( n 
N = NN 

no 500 J=1 ,N 
C< J 1=0.0 

KADD =< ( J-l )*J) /?. 

00 200 K = 1 » J 
KAOO“KAni) + l 

C < J 1 =C I J 1 *-A(KADD)*R(K) 

200 CONTINUE 
K 1 = J*1 

1F1K1-N) 250,250,500 
250 DO 300 K=Kl,N 
K A 0 0 = KADD«-K-1 
C { Jl= Cl JD A I K ADO 1 * B { K ) 
300 CONTINUE 
500 CONTINUE 
PE TURN 
END 



APPENDIX B 


A 



a . 

3 

t 

E {X} 

e{x|y} 

S’ (w) 

& [w(y) ,a] 
F 

f t (y |a.S) 


LIST OF SYMBOLS 

the space of allowable experiments 

the space of allowable experiments requiring ex- 

o 

actly j observations 

element of ' A 
fch 

the i experiment in A 

til 

experiment in A performed at the j stage of 
sampling 

vector of parameters appearing in combined model 
equations 

expectation of the random variable - X 
conditional expectation .of the random variable X 
given the value of Y 
entropy of the probabilities at state w 
entropy of the posterior probabilities if system is 
in state w and the value y is observed 
element of ^ 

sigma field of Borel sets over SI. 

, y m . 

density function of y under model £ when a is 
given and experiment a is to be performed, 
marginal density function of y under model £ 
when experiment a is to be performed. 
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density function of y under model i c when a 


*-y 

and a are given 

f jt ( ? j+ d a) 

marginal density function of y under model Z 

. J * 1 ■** * 


when a is given 

s i (w l w i-l) 

density function of w given w ^ 

H l 

denotes hypothesis Z about the form of the model 

o 

equation 

h 1 (z 1> ...,z k ) 

function of controlled variables defining x. 

* e I (w,a) 

expected information in experiment a when state 
of system is w 

I(w,a,i, j) 

expected information for discriminating in favor 
of 1L against in experiment a when state 

of system is w 

i 

denotes subscript of hypothetical model with larg- 
est posterior probability 

if. 

i 

true model equation number 

J MAX 

upper limit on total number of observations 

k(j) 

superscript of experiment performed at stage j of 
sampling 

L 

number of model equations or hypotheses postulated 

M 

design matrix 

\ 

design matrix for model l 

N(A) 

number of elements in A 

N(?,T) 

normal distribution with mean vector and pre- 


cision matrix T 
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n 


vector of n 


n i 

n(i, j) 

Pr (X) 
Pr{X | Y } 


Q,Q r Q 2 

R(w,a) 


j 


T 

W 

w 

X 


number of observations taken at. stage i 
number of times experiment is performed in j 

stages 

probability of event X 

probability , of event X given event Y 
c (i ) 

limiting proportion of times a performed in an in- 
finite sequence of experiments 
denote quadratic forms 

precision matrix of distribution of under model Z 

expected reduction in entropy if experiment a is per- 
formed and state is w 

mean vector of distribution of y, under model l 

J 

precision matrix of distribution of e * 
random variable defined over ft 
element of ; ft. An observed value 
vector of x, 


x, 

i' 


a , 

-V 

y>y 


ot. 


value of h^(z^, . „ . ,z^.) 


, th 


value of * * * yZ v) at ^ 

observed variable 
controlled variable i. 

vector of parameters in model equation Z 


Si 




coefficient of x. 

i 


coefficient of x^ in model £ 



defined on page 41 
Krone eke r delta function 
vector of observation errors 
probability model i is correct 

posterior probability that model i is correct after j 
stages of sampling 
stopping probability 

mean vector of distribution of parameters in model 
density function of parameters in model £ after j 
stages of sampling 
precision of distribution of £ 

precision matrix of distribution of parameters of model i 
after j stages of sampling 
state space of process. Defined on page 19 
direct product of state space and observed variable space 
vector of zeros 
proportional to 
determinant of a^matrix 
approximately equal to 
distributed as 


includes 



APPENDIX C 


TABLES 


TABU- i - SUMMARY OF SIMULATION RESULTS PKKsKNTKD TN TABLES 2 THROUGH 9 


Mod e 1 

Parameter 

L = 4, 

i* = 2 

L ■= 4, 

* 

1=3 

L * 6, 

A 

\ » 3 

I. = 6, 

A 

i = 5 



100 

«» 


‘'"AH 

100 

500 

100 

500 



obs 


WSM 


obs 

Obs 

obs 

obs 

mm 

MUM 

0 

0 

0 

0 

0 

0 

0 

0 



.966 

.983 

0 

0 

0 

0 

.010 

0 


iSmK bflEn&i 

.032 

.016 

.922 

.962 

.860 

.877 

.902 

.941 


JMSfe 

i002 

.001 

.078 

.038 

.109 

. 107 . 

.062 

.023 

H 






.022 

.012 

.017 

.029 

mm 





.009 

.004 

.009 

.006 


8 0 

0 0971 

0.0S91 

0.1322 

0.1377 

0.1116 

G. 1357 

0.02/8 

-0.0240 


6L 

0- 1010 

0.0951 

0. 1313 


0.1253 

0.1286 

0.0316 

0.0382 

■ 

•: 

.4981 

. 5025 

.2485 

| 

.2411 

. 2544 

.5114 

.4977 


So 

0.1013 

0.0941 

-0.0070 

-0.0067 

-0.0038 

-0.0015 

-0.0249 

-0.0099 



.4981 

.5025 

.2478 

.2523 

.2493 

. 2505 

. 50S6 

.5032 


fi 1 

s 2 

-.0007 

.0021 

.2578 

. 2634 

.2157 

2544 

.1179 

.0981 

mmm 

So 

0.1010 

0-0943 

-0.0070 

-0.0335 

-0.0035 

-0.0015 

-0.0237 

-0.0097 


B? 

.5029 

.4915 

.2497 

.2416 

.2372 

. 2429 

.5168 

.5046 


4 

-.0004 

.0018 

. 2579 

. 2634 

. 264 5 

.2544 

. 1168 

.0976 


e 3 

-.0076 

.01.12 

-.0026 

.0146 

- . 0089 

.0102 

-.0083 

-.0017 


a,, 





0.0040 

0-0015' 

-0.0157 

0.0001 







.2564 

.2440 

. 5120 

.5009 







. 2222 

. 2577 

-.0020 

.0227 


6 2 

e 3 

c 4 





.0032 


-.0048 

.0024 






.0360 

-.0015 

. 0908 

.0730 

6 

6. 





0.0030 

-0.0037 

-0.0180 








.2390 

.2559 

. 5057 

.4638 







.2291 

.2619 

.0603 

.0376 


< 





.0596 

-.0373 

.0438 

. 1340 


? 





.0301 

-.0050 

.0353 

. 059S 


»■ 





-.0534 

.034 5 

- . 04 1 3 

-.0969 


The column headings give the values of L and i* and the number of observations. The row headings 
present the parameters whose average posterior values are given. The probabilities listed for 100 ob- 
servations are the averages after five simulations of 100 observations and the values after r be first 
100 observations ot the 500 observation simulations. The averages of the posterior parameter means 
are based only upon the five full simulations of 100 and 500 observations, respectively- The posterior 
probabilities for 500 observations are based upon five simulations of 500 observations each. 



























































no 


TABLE '2 . 


L = 4, 


* 

i = 2 


Model 

Param 

After 100 observations 

After first 100 of 500 obser 


m 

0 1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Kill 

9 2 

.973 

■ .979 

.974 

.976 

.975 

.976 

.976 

.931 

.977 

.923 

mw 

°3 

.025 

.019 

.024 

.023 

.024 

.023 

.023 

.063 

.022 

.071 


e 4 

.002 

.001 

.002 

.001 

.002 

.001 

.001 

.006 

.001 

.006 


So 

0.0795 

0. 101.7 

0.0906 

0.1271 

0.0865 

* 


| 


* 

2 

e o 

0. 1187 

0.1017 

0.0753 

0.1032 

0.1059 



■ 




B 1 

.5192 

.5022 

.4935 

.6892 

.4865 






3 

e o 

0.1263 

0. 1021 

0.0682 

0.1067 

0.1033 




* 

■ 


B, 

.5191 

.5021 

.4935 

.4894 

.4866 


■ H 

■ 


H 


4 

-.0152 

- . 0008 

.0141 

-.0069 

.0055 


H 

| 


■ 

4 

^0 

0.1239 

0.1022 

0.0688 

0.1059 

0. 1041 

A 


■ 

* 



6 i 

.4858 

. 5044 

. 4771 

.5186 

.5288 







S 2 

-.0126 

-.0010 

.0133 

-.0051 

.0032 







3 3 

.0351 

-.0024 

.0170 

-.0350 

-.0527 







P 0 

0.25 

0.23 

0.23 

0.17 

0.19 

0.18 

0.23 

0.27 

0-21 

0.25 


Pi 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


P 2 

.05 

.05 

.03 

.17 

.20 

.24 

.06 

.04 

.02 

.01 


P3 

.02 

.03 

.02 

.07 

0 

.02 

.01 

.01 

.03 

0 


P4 

.35 

.20 

. 14 

.01 

.17 

.04 

.26 

.43 

.11 

.25 


P 5 

.06 

.20 

.27 

.14 

.06 

. 12 

. 14 

.01 

.19 

.24 


p 6 

.01 

0 

.06 

.09 

.02 

.03 

.01 

.01 

.17 

0 


P 7 

.05 

.05 

0 

. 13 

.21 

.19 

.06 

.02 

.01 

.01 


p 3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


p 9 

.21 

.24 

.25 

.22 

. 15 

.19 

.23 

.21 

.26 

.24 


Not recorded. 


The values of the posterior probabilities and parameter means after ten simulations, of 
100 observations each, of the sequential selection procedure. The last five columns are 
data from the first 100 observations of the 500 observation simulations tabulated in 
table 3. The posterior means were not recorded for tltise cases. Also listed are the 
proportions p^ of the times each a^i) was chosen as the optimal experiment. 
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TABLE 3 . - L = 4 , i* = 2 


Model 

Pa ram 

After c 500 observations 

1 

01 

0 

0 

0 

0 

0 

2 

e 2 

. 991 

.985 

.990 

.991 

.957 

3 

0 3 

.009 

.015 

.009 

.009 

.040 

4 

®4 

0 

0 

0 

0 

.003 

1 

e o 

0.0875 

0.0599 

0.0905 

0.0757 

0.1317 

2 

^0 

0.0921 

0.0984 

0,0999 

0.0942 

0.0909 


*1 

.4964 

< 5010 

.5028 

.5014 

.5108 

3 

3 0 

0.0923 

0.1032 

0.0984 

0.0937 

0.0827 


h 

.4964, 

.5010 

.5028 

.5014 

.5108 


e 2 

-.0005 

-.0096 

.0029 

.0012 

.0163 

4 

% 

0.0923 

0.1022 

0.0985 

0.0935 

0.0850 


% 

.4948 

.4886 

.5043 

.4882 

.4817 


^2 

-.0004 

- o 0086 

.0028 

.0014 

.0139 


% 

.0016 

.0126 

-.0017 

.0141 

.0293 


P 0 

0.234 

0.264 

0.236 

0,236 

0.228 


Pi 

0 

0 

0 

0 

.002 


Vo 

.050 

.010 

.056 

. 044 

0 


P3 

.008 

0 

.004 

.008 

0 


P4 

.280 

* 422 

.302 

.306 

.076 


P 5 

.110 

.060 

.070 

.088 

.418 


P6 

.018 

0 

.026 

.076 

.006 


P? 

.072 

.010 

.088 

. 038 

.002 


- P8 

0 

0 

0 

0 

0 


P9 

.228 

.226 

.218 

.204 

.268 | 


The values of the posterior probabilities and parameter 
means after 5 simulations, of 500 observations each, of 
the sequential selection procedure* Also listed are 
the proportions p^ of the times each a^) was chosen 
as the optimal experiment* 
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TABLE 4. - L = 4 , i* =3 


Rpppi 

farara 

After 100 observations 

After f 

irst 10 

0 of 50 

0 obser 

vat ions 

| 

n 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

mm 

.788 

.828 

.967 

° .966 

.966 

.962 

.916 

.966 

.941 


m 


.21.2 

. 17? 

.033 

.034 

.034 

-038 

.084 

.034 

.059 


■ 


0. 1140 

0- 3548 

0.1260 

0. 1292 

0-1368 

II 

* 

HR 


* 


E o 

Si 

0. 1183 

0.1515 

0.1255 

0. 1236 

0. 1.324 


* 

■ 

■ 

£ 


. 2452 

.2533 

.2385 

. 2575 

. 2482 





■ 

H 

&0 

-0-0043 

-0.0089 

-0.0159 

0.0104 

-O.OL62 

H 

* 

H 

1 

VMfl 



.2467 

.2510 

. r ■ 

.2552 

. 2470 






m 

4 

. 2263 

.2939 


.2216 

.2788 








-0.0045 

-0-0091 

-0.0158 

0.0103 

-0.0162 


* 


1 

1 

■ 


.1833 

.309 5 

.2395 

.2633 

. 2523 


- 


■ 




.2268 

.2945 

.2681 

. 2215 

.2788 






■ 

. OS33 

-.0779 

-.0009 

-.0107 

-.0070 



1 1 


MB 


Po 

Pi 

p 2 

P 3 

P4 

^5 

0. 18 

0.17 

0.17 ' 

0.17 

0.17 

0.18 

0. 17 

0.17 

0.17 - 

0.17 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


.32 

.31 

.30 

.30 

.29 

.32 

.30 

.30 

.31 

.30 


0 

.03 

0 

..02 

0 

0 

0 

0 

.01 

.02 


0 

0 

.03 

0 

.02 

0 

.03 

.02 

0 

0 


.03 

.01 

.02 

.03 

.04 

.02 

.02 

.02 

-.01 

.05 


0 

0 

.01 

.02 

0 

.01 

0 

0 

0 

.02 


p 6 

?7 

P£ 

P 9 

.30 

.32 

.30 

.28 

.30 

.30 

.31 

.31 

.32 

.28 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


. 1.7 

.18 

.17 

.18 

.18 

.17 

.17 

.18 

.18 

. 16 


The values of the. posterior probabilities and parameter means after 10 simulations, of 100 ob- 
servations each, of Lhe sequential selection procedure. The last 5 columns are data from the 
first 100 observations of the 500 observation simulations tabulated in table 5. The posterior 
means for these 5 cases were not tabulated. Also listed are the proportions p L of the times 
each was chosen as 'the optimal experiment. 
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TABLE 5. - L = 4, 1* = 3 


Model 

Pa ram 

After 500 observations 

1 

0 i 

0 

0 

0 

0 

0 

2 

0 2 

0 

0 

0 

0 

0 

3 

e 3 

.953 

.982 

.953 

-.969 

.954 

4 

S 4 

.047 

.018 

.047 

.031 

, 046 

1 

"CD 

O 

0.1368 

0.1385 

0.1383 

0.1394 

0.1354 

2 

3 0 

0.1376 

0.1398 

0.1391 

0,1387 

0.1369 


e i 

.2561 

° .2463 

.2608 

.2550 

.2426 

3 

»0 

-0.0227 

0.0085 

-0.0069 

-0.0028 

-0.0096 


e l 

.2566 

,2469 

. 2611 

.2546 

.2434 


e 2 

. 2906 

.2392 

.2648 

.2548 

. 2677 

4 

s o 

-0.0227 

0.0085 

-0,0069 

-0.0028 

-0.0096 


h 

.2355 

. 2385 

.2399 

.2715 

.2225 


^2 

.2905 

.2392 

. 2649 

.2548 

.2678 


e 3 

.0281 

.0112 

.0281 

-.0223 

.0277 


PO 

0.178 

0.178 

0.178 

— 

0.178 

0.178 


Pi 

0 

0 

0 

0 

0 


P 2 

. 3.2.2 . 

.320 

.320 

.318 

.3-18 


p 3 

0 

0 

0 

.002 

. .004 


p 4 

0 

.006 

.004 

0 

0 


P5 

.004 

.004 

.004 

.002 

.010 


P6 

.002 

0 

0 

0 

.004 


P? 

.318 

.318 

.318 

.320 

.312 


p 8 

0 

0 

0 

0 

0 


P 9 

.176 

i 

.174 

. 176 

oo 

o 

.174 


The values of the posterior probabilities and parameter means 
after 5 simulations, of 500 observations each, of the se- 
quential selection procedure. Also listed are the propor- 
tions p i of the times each a(i) was chosen as the optimal 
experiment. 
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TABLE 6. -I. = 6, i*“3 


Mod c 1 

Param 

After 100 observations 

Af ter 

first 100 of 500 

observari ons 

mm 

6 1 

0 

0 

0 


0 

0 

0 

0 

0 

0 

n 

0, 

wmw 

0 

0 

0 

0 

0 

0 

0 

0 

0 

mm 

P 

0/ 


.9564 

.7630 

.9093 

.943? 

. 2733 

.9534 

.9471 

.9449 

. 9354 

MM 


.0365 

.1756 


.0469 

. 5534 

.0400 

.0444 

.0457 

.03 VS 

5 

4 

.0052 

.0055 

.0453 

.0130 

.0075 

. 1168 

.0050 

.0070 

.007 5 

.005? 

6 

°6 

.0016 

.0017 

.0162 

.0034 

.0024 

.0364 

.0016 



.00)5 

.0019 

.0016 


TO 

O 

0.1567 

0.1026 

0.0891 

0.0839 

0- 1259 

* 

mM 






0.1298 

0. 1197 

0. 1192 

0.1118 

0.1462 

■ 

■ 

BHI 



II 

.2696 

.2258 

.2131 

. 2422 

. 2527 

MM 

b - 



!■ 

3 


-0.0114 

0.0098 

-0.0332 

0 . 0004 

0.0356 

■ 

■9H 



* 


h 

.2564 

. 23 54 


.2542 

.2607 


HUB 





*2 

. 2882 

.2282 


.2329 

. 2463 


1 




4 


-0.0110 

0.0098 

-0.0317 

-0.0008 

0.0161 

* 

* 

* 

* 

* 


S 1 

.2421 

. 2427 

.3033 

.2117 

.2864 








.2877 

.2282 

.3268 

.2346 

KBI 







83 

.0189 

-.0099 

-.0846 

.0562 

Rjj 






5 


-0.0056 

0.0200 

-0.0082 

-0.0175 

0.0314 

■ 



■ft - 

* 



.2458 

.2403 

. 2870 

.2232 

. 2855 


H 





8 2 

.2562 

.1692 

.1788 

.3427 

.1641 

H 

H 

H 




63 

.0148 

-.0080 

-.0676 

.0432 

.0336 

H 


■ 




p '4 

.0271 

.0512 

.1288 

-.0966 

.0694 

1 


■ 


■ 

6 

e o 

-0.0061 

0.0209 

-0.0130 

-0.0166 

0-0300 


■ 

■ 

* 




. 2429 

. 1945 

.3250 

.2050 

.2278 


■ 

H 




.2583 

. 1767 

.1921 

.3418 

.1767 

H 


H 




83 

.0265 

. 1913 

-.2368 

.1214 

. 1958 



■ 


H 


H 

.0254 

.0432 

.1203 

-.0968 

.0584 


H H 

H 


H 


B 5 

-.0089 

-.1555 

.1320 

-.0609 

-.1739 


| 1 



n 


H 

0.13 

0.15 

0. IS 

0.17 

0.17 

0.18 

0. L0 

0-11 

0. 17 

0.13 


.02 

.08 

.03 

.06 

. 10 

.01 

.04 

. 1 1 

. 1 2 

.02 


. 17 

.24 

.32 

.27 

.22 

. 3/. : 

- 12 

. 1 2 

. 20 

. 17 


.08 

.03 

0 

0 

.02 

0 

.05 

.02 

.02 

.08 


.06 

.03 

.02 

.07 

.03 

.01 

.07 

. 14 

.07 

.06 

■ 

P5 

P6 

P? 

P8 

P9 

.03 

.03 

.07 

.03 

.05 

0 

. 11 

.02 

.04 

.03 

lllliiiS 

0 

. 12 

. 14 

.14 

.02 

.06 

.02 

.06 

.07 

0 

■ ■ 

.29 

. 17 

. 11 

.09 

.21 

.26 

.16 

.15 

. 15 

.29 

■ 

.05 

.02 

.01 

.05 

.04 

0 

. L6 

. 1 2 

.04 

.05 

■ 

. 17 

.13 

.12 

.12 

. 1.4 

.16 

.17 

. 1 5 

. 12 

. 17 


* 

Noc recorded. 


The values of the posterior probabilities and parameter means after 10 simulations, of 100 obser- 
vations each, of the sequential selection procedure. The last 5 columns are data from tlu* first 
100 observations. of the 500 observation simulations tabulated in table 7. The posterior means 
were not recorded for these 5 cases. Also listed are the proportions of the times eacli a ( * > was 
chosen as the optimal experiment. 
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TABLE 7. - L = 6 , i* - 3 


Mod el 

Fa ram 

After 500 observations 

1 

6 i 

0 

0 

0 

0 

0 

2 

e 2 

0 

0 

0 

0 

0 

3 

63 

.6046 

.9812 

.9722 

.9746 

.8526 

4 

e 4 

.3388 

.0175 

.0257 

.0230 

. 1316 

5 

Or 

.0425 

.0009 

.0018 

.0021 

.0125 

6 

*6 

.0141 

.0003 

.0003 

. 0003 

.0032 

1 

h 

0.1321 

0.1378 

0.1325 

0. 1349 

0.1413 

2 

So 

0.1356 

0.1278 

0.1247 

0.1168 

0. 1383 



.2434 

.2616 

.2541 

. 2549 

. 2581 

3 

e 0 

-0.0026 

0.0067 

-0.0066 

- 0.0021 

-0.0029 


6 1 

.2446 

.2556 

.2486 

.2466 

.2571 


e 2 

.2542 

. 2336 

.2605 

.2615 

.2622 

4 

e 0 

-0.0027 

0.0067 

-0.0067 

-0.0017 

-0.0030 


6? 

.2076 

.2515 

.2340 

. 2341 

.2872 


8 2 

. 2544 

.2335 

. 2608 

.2609 

.2623 


S3 

.0491 

.0055 

.0194 

.0167 

-.0399 

5 

» e 0 

-0.0113 

0.0077 

0.0111 

0.0063 

-0.0062 



.2081 

.2518 

.2332 

. 2394 

.2873 


h 

.2926 

.2279 

. 2845 

.2061 

. 2774 


£ 

.0486 

.0052 

.0203 

.0101 

-.0401 


< 

, -.0300 

.0049 

-.0200 

.0496 

-.0121 

6 

% 

-0.0118 

0.0047 

- 0.0100 

0.0061 

-0.0073 


h 

.2164 

.3128 

. 2432 

. 2386 

. 2682 


4 

.2949 

.2455 

. 2801 

.2070 

.2819 


63 

.0130 

-.2307 

-.0225 

.0134 

.0405 



-.0317 

-.0098 

-.0168 

.0490 

-.0155 


^5 

.0270 

. 1773 

.0328 

-.0026 

-.0618 


Po 

0.178 

0.136 

0.148 

0.116 

0.168 


Pi 

.004 

.064 

.028 

.070 

.012 


X 

P? 

.314 

.206 

.198 

.086 

.282 


P 3 

.004 

.036 

.092 

.112 

.018 


p 4 

.010 

.014 

.038 

. no 

.014 


P 5 

0 

.092 

.014 

.024 

.016 


P6 

.022 

.004 

.012 

.022 

.002 


P 7 

. 296 

.190 

. 280 

.288 

.304 


PS 

0 

. 104 

.026 

.014 

.010 


Pg 

. 17 2 

.154 

.164 

's/I 

CO 

. 17 4 






1 



The values of the posterior probabilities and parameter 
means after five simulations, of 500 observations each, of 
the sequential selection procedure. Also listed are the 
proportions of the times each a (0 was chosen as the opti- 
mal experiment. 
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TABLE 8. - L = 6, i* = 5 


Model 

Pa ram 

After 100 observations 


. 

nations 

1 

C 1 

0 

0 ' 

0 

0 

0 

0 

0 

0 

0 

0 

7 

0 

0 

0 

.021 

.04 2 

0 

0 

.01 1 

.003 

.026 

3 

0 3 

.94 5 

.942 

.956 

.895 

,84 8 

.943 

.877 

.852 

.904 

.857 

4 

°4 

.043 

.043 

.038 

.063 

.04 2 

. 048 

.106 

.089 

.070 

.076 

5 

ot 

.007 

.012 

.005 

.015 

.033 

.007 

.013 

.03 5 

.018 

.026 

6 , 


-004 

.003 

.001 

.006 

.035 

.002 

.004 

.013 

.006 

.013 

1 

s o 

0. 1502 

-0.0299 

0.0231 

-0.0079 

0.0034 

* 

‘ft 

■ 

1 


■ 

"1 

0.0356 

0.O1S9 


0.0316 

0.0431 






■ 

.5159 

. 5101 

. 5123 

.5079 

. 5106 



1 




B o 

-0.0348 

-0.0413 

- 0.0412 

-0.0098 

0.0026 

£ 

| 

* 




Bl 

.5040 

.507 7 

.5096 

. 5084 

.5133 







e 2 

.1467 

.1265 

.1478 

.0837 

.0850 

■ 








-0.0333 

-0.0396 

-0.0414 

-0.0070 

0.0026 

■ 

■ 

■ 




4 
6 7 

.4874 

.5288 

. 5019 

.5513 

.5146 


1 





. 1450 

. 1 252 

.1478 

.0810 

.0849 





■ 

■ 


.0217 

-.0261 

.0099 

-.0456 

-.0016 

Hi 



1 

■ 


6 0 

-0.0232 

-0.0242 

-0.0431 

-0.0041 

0.0161 

■ 


H 

1 

* 


4 

p> 2 

.4985 

.5201 

.5039 

.5516 

.4861 




■ 



.0642 

-.0063 

. 1616 

.0184 

-. 1260 



■ 

■ 



g 3 

.0101 

-.0185 

.0074 

-.0463 

.0235 



■ ■ 

■ ■ 



p 4 

. 07 74 

.1223 

-.0131 

.0605 

.2068 




| | 

■ 

6 


-0.0298 

-0-0234 

-0.0435 

-0.0067 


■ 

* 

j 




4 

&2 

.6005 

. 5398 

.5233 

.5143 


■ 






.1187 

-.0120 

.1655 

.0517 




H 




S 3 

a 4 

-.3886 

-.1048 

-.0708 

.1797 

.6034 




■ 



.0292 

.1270 

-.0167 

.0301 

. 1071 





illaMllB 


5 5 

.3037 

.0673 

.0599 

-.1893 

-.4482 





MBH 


PO 

Pi 

Pt 

0.10 

0-16 

1 

0.24 

0.18 

0.18 

0.14 

0.17 

0.18 

0.22 


.05 

.06 

1 1 

.02 

.13 

.25 

.04 

.02 

.01 

.01 

■ 

.08 

.20 

. I %: 

.03 

.03 

.06 

.17 

.04 

.25 

.02 

■ 

P3 

P4 

P5 

P. 

.04 

.05 


.07 

.02 

.01 

.06 

.01 

.03 

0 


.08 

.13 

.08 

.26 

.18 

.16 

.06 

.10 

. 10 

.05 


.OS 

.08 

.10 

.12 

-19 

.07 

.17 

-25 

.06 

.41 


.08 

.04 

.06 

.02 

.03 

.03 

0 

0 

. 14 

.01 

jffl 1 

r 6 

p? 

P8 

p 9 

. 11 

.10 

.09 

.03 

.05 

.09 

. 19 

.19 

.05 

.02 

lafl . 

.22 

.02 

. 10 

0 

.01 

.06 

.03 

.01 

.01 

0 


.16 

.16 

.14 

.21 

.18 

.09 

.14 

.21 

. 17 

.26 


Not recorded. 


The values of the posterior probabilities and parameter means after 10 simulations, of 100 ob- 
servations each, of the sequential selection procedure. The last 5 columns are data from Lhe 
first 100 observations of the 500 observation simulations tabulated in table 9. Also listed 
are the proportions of the times each a< 1 ) was chosen as the optimal experiment. 
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TABLE 9 . - L = 6 , i 5 ' ~ 5 


Model 

Par am 

After 500 observations 

1 

°1 

0 

0 

0 

0 

0 

2 

e 2 

0 

0 

0 

0 

0 

3 

a 3 

.974 

.882 

.899 

.976 

.976 

4 

- °4 

.020 

.024 

.029 

.021 

.022 

5 

G 5 

.003 

.07 5 

.062 

.002 

; 002 

6 

°6 

.002 

.020 

.009 

0 

0 

1 

8 0 

-0. 1051 

0.0255 

-0.0926 

0.0130 

0.0390 

2 

8 0 

0,0290 

0.0373 

0.0436 

0.0351 

0.0458 


*1 

.4860 

.5032 

.4903 

.5038 

. 5053 

3 

8 0 

-0.0181 

-0.0137 

0.0018 

-0.0258 

0.0065 


8 1 

. 5016 

. 5008 

.5019 

.5056 

. 5061 


8 2 

, 1075 

.1046 

.0803 

f— ■ 
CO 

.0791 

4 

8 0 

-0.0179 

-0.0142 

0.0027 

-0.0257 

0.0066 



. 5035 

.4859 

.5201 

.5177 

.4956 


b 2 

. 1071 

.1050 

.0782 

. 1187 

.07 90 


f 3 

-.0025 

.0198 

-.0237 

-.0160 

.0138 

5 

»0 

-0.0100 

0.0012 

0.0159 

-0.0202 

0.0136 


8 1 

.4959 

.4940 

.5016 

.5169 

.4962 


b 2 

.0413 

-.0152 

-.0328 

.0897 

.0306 


8 3 

.0061 

.0081 

0 

-.0151 

.0129 


< 

.0653 

.1185 

.1116 

.0241 

.0435 

6 

8 0 

-0.0166 

-0.0044 

0.0157 

-0.0186 

0.0135 


e i 

.4232 

.4352 

.4617 

.5039 

.4949 


6 2 

.0891 

.0094 

-.0250 

.0835 

.0309 


b 3 

. 2801 

.2087 

.1232 

.0400 

.0179 


Ba 

.0224 

.0996 

.1030 

.0288 

.0452 


8 5 

-.2070 

-.1452 

-.0854 

-.0430 

-.0039 


p 0 

0. 158 

0.106 

0.208 

0.174 

0.13 2 


Pi 

.252 

.190 

.314 

.010 

.128 


P 2 

.108 

.054 

.016 

.306 

.138 


P 3 

.040 

.052 

.014 

.006 

.008 


P4 

.170 

.184 

.100 

,020 

.104 


P 5 

.022 

.034 

.114 

.030 

.088 


P B 

.056 

0 

0 

.07 2 

.002 


p 7 

.074 

. 134 

.052 

.208 

.172 


p 8 

.036 

.118 

.110 

.022 

.092 


p 9 

.084 



.128 

.072 

. 152 

. 132 


The values of the posterior probabilities and parameter 
means after five simulations, of 500 observations each, of 
the sequential selection procedure, Also listed are the 
proportions of the times each a ( * ) was chosen as the op- 
timal. experiment. 
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TABLE 10* - LARGE SAMPLE STUDY TWO 


Hodt 1 

Param 


*0 

- (0.1,0. 2, 0.3,0. 

4) 



o 0 • 

(0.25,0, 

25,0,25,0.25) 



V < C 

.4, 0.3,0. 

2 , 0 . 1 ) 





5 simulations 

Average 


5 

simulations 


Average 



5 elmulat 

ions 


Average 

i 

9 1 

0 

0 

0 

on 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

B 2 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

B 3 

.937 

.947 

.328 


.960 

.924 

.969 

.969 

.966 

.967 

.930 

. 960 

.984 

.979 

.976 

.957 

.984 

.976 

4 


.043 

.053 

.172 

.074 

.040 

.076 

.031 

.031 

.034 

.033 

.070 

.040 

.016 

.021 

.024 

.043 

.016 

.024 

1 

B 0 

-0-0338 

-0-0460 

.0015 

-0.0282 

-0.01 25 

-0.0238 

-0.0413 

-0.0230 

-0.0204 

-0.0501 

-0.0172 

-0.0304 

0.0392 

0.0023 

. 

0.0045 

-0,0195 

0.0180 

0.0090 

2 

p 

-0.033S 

-0.0460 

0.0015 

-0.0282 

-0.0125 

-0.0238 

-0.04.13 

-0.0230 

-0.0204 

-0.0501 

-0.0172 

-0.0304 

0.0392 

o.oo2e 

0.0045 

-0.0195 

0.0180 

0.0090 


h 

1.0044 

.9810 

.9938 

.9989 

.9644 

.9885 

1.0142 

.9779 

1.0510 

1.0487 

.9697 

1.012 

.9484 

1.0213 

1.0603 

.9906 

.9614 

.9964 

3 

6 0 

-0.0333 

-0.0460 

0.0015 

-0.0262 

-0.0125 

-0.0238 

-0.0413 

-0.0230 

-0.0204 

-0.0501 

-0.0172 

-0.0304 

0.0392 

0.0028 

0.0045 

-0.0195 

0.0180 

0.0090 


e l 

1.0044 

.9810 

,9938 

.9989 

.9644 

.9885 

1.0142 

. 9779 

1.0510 

1,0487 

.9697 

1.012 

.9434 

1.0213 

1.0603 

.9906 

,9614 

.9964 


4 

-1.0555 

- 1.0012 

-.9594 

-1.0026 

-.9948 

-1.003 

-.9753 

-1.0205 

-1.0208 

-1,0139 

-.9762 

- 1.001 

-.9451 

-.9539 

-1.0253 

-.9339 

- 1.0012 

-.9719 

4 

e o 

-0.0338 

-0.0460 

0.0015 

-0.0282 

-0.0125 

-0.0238 

-0.0413 

-0.0230 

-0.0204 

-0.0501 

-0.0172 

-0.0304 

0.0392 

0.0028 

0.0045 

-0.0195 

0.0180 

0.0090 


61 

1.0044 

.9810 

.9938 

.9989 

.9644 

.9885 

1.0142 

.9779 

1.0510 

1.0487 

.9697 

1.012 

.9484 

1.0213 

1.0603 

.9906 

.9614 

.9964 


e 2 

-1.0555 

- 1.0012 

-.9594 

-1.0026 

-.9948 

-1.003 

-.9753 

-1.0205 

-1.0208 

-1.0139 

-.9762 

- 1.001 

-.9451 

-.9539 

-1.0253 

-.9339 

- 1.0012 

-.9719 


$ 3 

.0110 

-.0236 

.0564 

.0355 

-.0016 

.0155 

-.0012 

-.0057. 

-.01 52 

-.0121 

.0418 



.0016 

.0019 

-.0239 

.0290 

-.0456 

0. 0087 

-.0060 


The valuer of the posterior probabilities and parameter means after five simulations, of 500 observations each, of the sequential selection procedure with three different 
pr K r dis tributions on the models- 
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TABLE 11. - LARGE SAMPLE STUDY THREE 


Model 

Parameter 

t = loo.o 



T = 

1.0 





T = 0 

.01 






5 simulations 


Avg 


5 s 

imulat ions 


Avg 


5 simulations 


Avg 

3 


1 

1 

1 

1 

1 

1 

1 

l 

l 

1 

L 

1 

0.9979 

0.8155 

0.9999 

0.8951 

0.9990 

0.9415 

2 

°2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

.0006 

.1843 

0 

.0012 

.0004 

.0373 

i 

°3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

.0015 

.0002 

.0001 

. 1037 

.0006 

.02)2 

i 

S 0 

0.953 

1.027 

1.094 

0.979 

1.106 

1.032 

1.111 

1.091 

1.046 

1.298 

0,932 

1.096 

1.415 

0.454 

1.221 

0.916 

0.906 

0.932 


S 1 

1.069 

1.046 

.972 

1.067 

.979 

1.027 

1.009 

.890 

.957 

.800 

.874 

.906 

1 0 G93 

1.274 

1.318 

.632 

i.166 

1.09 7 


B 2 

. 984 

1.039 

.997 

.907 

1.033 

.992 

.862 

.910 

1.210 

,822 

1.160 

.993 

1.163 

.542 

1 , 607 

1.119 

1.193 

1.125 

2 

O 

0 

1.230 

1 . 254 

1.298 

1.262 

1.295 

1.268 

1. 135 

1.075 

1.567 

1.281 

0.527 

l. 117 

1.413 

0.454 

0.754 

0.914 

0.904 

0.888 


e 2 

1.559 

1. 564 

1.469 

1.520 

1.494 

1.521 

1.030 

.911 

1.009 

.816 

.907 

.935 

l . 122 

L. 275 

1.319 

.673 

1.172 

1. 112 


S 3 

.770 

.746 

. 702 

.738 

. 705 

.732 

.039 

.019 

. 039 

-.020 

.219 

.059 

.074 

.156 

.031 

,201 

.018 

.096 

3 

»0 

1. 742 

1.774 

1.78$ 

1.754 

1.796 

.1.771 

1.122 

1.10S 

1 . 002 

1.306 

0.940 

1.096 

1.415 

0.4L8 

1.221 

0.916 

0.906 

0.975 



1.515 

1.559 

1.480 

1.438 

1.520 

L.502 

.882 

.928 

1.246 

.842 

1.159 

1.0L1 

L. 184 

.613 

1.607 

1.121 

1.198 

1. 145 



.258 

.226 

.212 

,246 

. 204 

.229 

-.068 



.013 

-.041 

-.048 

.057 

-.013 

.056 

.301 

. 00.5 

.08 6 

.012 

.092 


.Cumber of 

3 

3 

3 

3 

3 

3 

49 

47 

39 

59 

52 

49 . 2 








t r i ai.s until 








































The posterior probabilities and posterior parameter means after five simulations of the sequential selection procedure for three different values 
For i = 100.0 and t * 1.0 the number of trials until • =1.0 (within the accuracy of the computer) Is also tabulated. For r = 0.01 the 
values are based upon 1000 observations. 
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TABLE 12. - SMALL SAMPLE STUDY ONE (« 3 TRUE) 


[J 


MAX 


8 ] 


0 

m 

T 

U 3, 

0 

PCS 

ASN 

Starting value 
for random seq 

* 

0o 70 

0.5 

( 0 , 0 ) 


0.133 

6 , 

36 


* 



.70 

.5 

(0.5, 

0.5) 

<458 

7, 

15 

041- 

574 

501 

221 

.70 

.5 

(l.o. 

1 . 0 ) 

.544 

6 . 

82 

261 

404 

147 

531 

.70 

,5 

(1.5, 

1.5) 

.446 

5, 

89 

251 

233 

17 5 

021 

.80 

.5 

( 0 , 0 ) 


.173 

7. 

50 

265 

603 

111 

061 

.80 

,5 

(0.5, 

0=5) 

,468 

7 ... 

78 

237 

616 

233 

015 

.80 

.5 

( 1 , 0 , 

1 = 0 ) 

. 531 

1, 

52 

066 

231 

644 

355 

.80 

. 5 

(1.5, 

1 = 5) 

= 460 

1, 

24 

124 

715 

646 

251 

t 90 

.5 

( 0 , 0 ) 


.229 

7. 

98 

202 

255 

025 

241 

.90 

.5 

(0=5, 

0.5) 

■ .479 

7 » 

92 

020 

625 

75? 

465 

= 90 

.5 

( 1 . 0 , 

1 = 0 ) 

.513 

7, 

81 

154 

510 

176 

555 

.90 

,5 

(1.5, 

1.5) 

.439 

7,. 

76 

043 

355 

261 

141 

.70 

1,0 

( 0 , 0 ) 


.397 

5. 

49 

031 

264 

7 22 

101 

.70 

1.0 

(0.5, 

0.5) 

.673 

5 . 

88 

142 

153 

215 

611 

.70 

1-0 

(1.0, 

1.0) 

.737 

5. 

29 

025 

206 

250 

121 

.70 

1.0 

(1.5, 

1^5) 

.621 

4 « 

84 

244 

233 

735 

061 

,80 

1.0 

(0, 0) 


.558 

6. 

90 

337 

020 

177 

205 

.80 

1.0 

(0.5, 

0.5) 

. 7 55 

6, 

94 

361 

341 

044 

651 

. .8,0 

1.0 

(1.0, 

lc,0) 

.771 

6. 


231 

737 

436 

405 

.80 

1,0 

(1.5, 

1.5) 

.700 

6 0 

22 

107. 

152 

46 0 

271 

,90 

1.0 

(0, 0) 



7. 


316 

753 

345 

645 

.90 

1,0 

(0.5, 

0.5) 

.765 

7, 

45 

042 

264 

053 

551) 

.90 

1.0 

(l.o, 

1.0) 

.777 

7 . 

15 

304 

456 

707 

705 j 

.90 

1.0 

(1-5, 

1.5) 

.689 

7, 

12 

324 

670 

521 

455’ 

.70 

2.0 

(0, 0) 


.699 

4. 

24 

034 

773 

264 

025 

.70 

2.0 

c (0.5, 

0.5) 

.871 

4 = 

03 

361 

656 

711 

7 21 

.70 

2.0 

(1-0, 

1=0) 

.8 7? 

3 

62 

110 

151 

661 

121 

.70 

2.0 

(1.5, 

1.5) , 

.7 23 

3. 

48 

000 

766 

306 

641 

.80 

2.0 

(0, 0) 


.868 

5. 

45 

020 

542 

27? 

271! 

.80 

2.0 

(0.5, 

0.5) 

.962 

4 

99 

073 

755 

766 

635 

.80 

2.0 

(1.0, 

1.0) 


4, 

63 

013 

527 

071 

701 

.80 

2,0 

(1-5, 

1.5) 

<872 

4 

61 

041 

554 

522 

311 

.90 

2.0 

(0, 0) 


.944 

6 

46 

001 

231 

353 

331 

.90 

2.0 

(0.5, 

0.5) 

.967 

5. 

66 

361 

503 

2^5 

415 

,90 

2.0 

(l.o. 

1.0) 

.969 

5. 

48 

151 

650 

040 

041 

,90 

2.0 

(1.5, 

1.5) 

= 939 

5 = 

80 

233 

434 

565 

701 


*Not recorded. 


Resulting* PCS and ASN values for Umax ~ 8 * and the combinations 
of G m , x t and U 3 g. Results are based upon 1500 Simula ti. ons of 
the procedure for each combination. 
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TABLE 13 o - SMALL SAMPLE STUDY ONE (H 3 TRUE) 


0 

m 

T 

o 

. ^ 
t3- 

PCS 

ASN 

Starting value 
for random seq* 

0 o 70 

0.3 

(0, 0) 

0*354 

9.48 

272 

036 

225 

461 

.70 

.5 

( 0 . 5 , 0 . 5 ) 

.665 

10.7 

057 

343 

345 

741 

.70 

.5 

( 1 . 0 , 1 . 0 ) 

*723 

9*63 

073 

144 

502 

151 

o 70 

*3 

( 1 * 5 , 1 . 5 ) 

*555 

7,38 

231 

500 

657 

525 

.80 

*5 

(0, 0) 

*508 

13,6 

033 

254 

034 

051 

* 80 

.5 

( 0 . 5 , 0 , 5 ) 

.761 

13.3 

225 

553 

740 

341 

c 80 

*5 

( 1 . 0 , 1 . 0 ) 

.806 

12*3 

134 

537 

257 

6 51 

. 80 

* 5 

( 1 . 5 , 1 * 5 ) 

.661 

11.8 

* 251 

356 

646 

745 

.90 

.5 

( 0 , 0 ) 

,574 

15.5 

056 

53 ? 

424 

615 

e 90 

*5 

( 0 = 5 , 0 * 5 ) 

.752 

14*6 

246 

632 

674 

651 

.90 

.5 

( 1 * 0 , 1 . 0 ) 

.800 

13.9 

140 

077 

157 

311 

c . 90 

.5 

( 1 * 5 , 1 . 5 ) 

*710 

13*8 

044 

035 

362 , 

005 

.70 

1*0 

( 0 , 0 ) 

.548 

6*53 

173 

052 

463 

251 

.70 

1*0 

( 0 . 5 , 0 * 5 ) 

.821 

6*82 

063 

364 

104 

441 

.70 

1.0 

( 1 * 0 , 1 * 0 ) 

.825 

6*09 

233 

034 

770 

255 

.70 

1*0 

( 1 . 5 , 1 * 5 ) 

*637 

5*36 

" 017 

237 

125 

325 

o 8 0 

1*0 

( 0 , 0 ) 

.808 

9.48 

275 

264 

535 

015 

. 80 

1 . 0 ' 

( 0 . 5 , 0 , 5 ) 

.971 

9.30 

015 

352 

36 0 

531 

.....80 

. 1*0 

.(, 1 * 0 , 1 . 0 ) 

.961 

8 * 16 

017 

142 

770 

505 

c 80 

1.0 

( 1 . 5 , 1 * 5 ) 

.86 5 

7 *86 

004 

724 

275 

765 

.90 

1*0 

( 0 , 0 ) 

*927 

12*1 

161 

027 

043 

101 

.90 

1*0 

( 0 . 5 , 0 . 5 ) 

*973 

10.8 

101 

732 

737 

651 

■>90 

1*0 

( 1 . 0 , 1 . 0 ) 

*964 

10.1 

016 

351 

614 

135 

.90 

1*0 

( 1 * 5 , 1 * 5 ) 

.958 

10*6 

171 

716 

57 2 

235 

*70 

2.0 

( 0 , 0 ) 

.700 

4.25 

073 

021 

660 

321 

*70 

2.0 

( 0 . 5 , 0 . 5 ) 

.878 

4.17 

003 

466 

340 

375 

.70 

2*0 

( 1 . 0 , 1 * 0 ) 

.855 

3.59 

337 

170 

131 

645 

*70 

2.0 

( 1 . 5 , 1 . 5 ) 

.714 

3.51 

055 

666 

256 

215 

*80 

2*0 

( 0 , 0 ) 

.911 

5*67 

037 

537 

412 

725 

.80 

2.0 

( 0 . 5 , 0 . 5 ) 

.990 

5.12 

111 . 

525 

350 

761 

*80 

2*0 

( 1 . 0 , 1 . 0 ) 

*988 

‘ 4.84 

003 

413 

673 

201 

.80 

2.0 

( 1 . 5 , 1 . 5 ) 

.894 

4*71 

055 

643 

644 

455 

.90 

2.0 

( 0 , 0 ) 

.996 

7.13 

374 

543 

153 

375 

.90 

2.0 

( 0 . 5 , 0 * 5 ) 

1.00 

6.25 " 

133 

225 

727 

441 

*90 

2.0 

( 1 . 0 , 1 . 0 ) 

1*00 

5.94 

332 

405 

117 

171 

.90 

2.0 

^ 1 * C 3 ^ 1 l 1 3 ) 

.995 

6.20 

010 

312 

536 

461 


Resulting PCS and ASN values for Umax = 16 and the combi natl ons 
of 0 m , t, and q* Results based upon 1000 simulations. 
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TABLE 14, - SMALL SAMPLE STUDY TWO (H 2 TRUE) 


t J MAX ■ 81 


0 

m 

T 

y 2,0 

PCS 

ASN 

Starting value 
for random seq. 

0,70 

0.5 

(1.0) 

0.760 

7.86 

052 

516 

237 

355 

,80 

.5 

(1.0) 

.734 

7.98 

016 

160 

602 

721 

o 90 

.5 

(1,0) 

.740 

7.98 

245 

577 

171 

655 

.70 

1.0 

( * 5) 

.828 

7.63 

321 

722 

414 

631 

o 7 0 

1.0 

(1.0) 

.882 

7,20 

340 

321 

470 

071 

.70 

1.0 

(1.5) 

,80 0 

6.86 

360 

415 

546 

645 

.80 

lo0 

(1.0) 

.872 

7,98 

273 

760 

237 

431 

.90 

1.0 

(.5) 

.880 

7,97 

006 

761 

404 

3 25 

,90 

1,0 

(1.0) 

,898 

7.98 

331 

151 

347 

271 

.90 

loO 

(1.5) 

c 832 

7.99 

372 

024 

174 

Oil 

,70 

2o 0 

(1.0) 

.900 

5.13 

004 

415 

604 

245 

.80 

2 o 0 

(1.0) 

.936 

j 7.89 

063 

456 

575 

211 

,90 

2.0 

(1,0) 

.934 

| 7.98 

065 

654 

616 

225 


The PCS and ASN values resulting from 500 simulations of the 
sequential procedure for each of the tabulated combinations 
of e m> . t, and ? 2>0 . 



u 

m 

T 

^3,0 

PCS 

ASN 

Starting value 
for random seq. 

0. 70 

1.0 

(0, 1, -1) 

0.767 

7.55 


* 



.70 

1.0 

(0, 0.5, 0) 

. 442 

7.10 

006 

171 

767 

411 

.70 

loO 

(0, 0, 0.5) 

.027 

5.74 

113 

071 

707 

045 

*70 

1.0 

(1, 0.5. 0) 

.154 

6.40 

032 

457 

065 

345 

.80 

1.0 

(0, 1, -1) 

.792 

7.91 ; 

315 

037 

701 

221 

.80 

1.0 

(0, 0.5, 0) 

.524 

7.67 

070 

131 

010 

071 

.80 

1.0 

(0, 0, 0.5) 

c 030 

6.73 

044 

541 

754 

365 

.80 

1.0 

(1, 0.5, 0) 

.200 

7.31 

03 4 

26 4 

602 

535 

.90 

1.0 

(0, 1, -1) 

*790 

7.99 

175 

260 

740 

521 

.90 

loO 

! (0, 0.5, 0) 

.506 

7.91 

1 000 

24? 

732 

655 

.90 

1.0 

(0, 0, 0.5) 

.025 

7.51 

; 276 

504 

634 

101 

.90 

1.0 

(1, 0.5, 0) , 

.210 

° 7.81 

243 

240 

621 

255 


•k 

Not recorded* 


PCS and ASN values resulting from 1000 simulations performed for 
the indicated combinations of 6 m , t, and q ; 
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6 • 


TABLE 3 7. - 0F ANALYSIS OF EQUATION (7-1) 

££?$& DATA OF TABLE 16 



Eg t irr.# t'Z'l 
coefiicY-M 

t-statistic 

! 

Descriptive 

significance 

level 

0,112 

28.4 

0,999+ 

-,354E~.3 

6.8 

.999+ 

.323E-1 

3,4 

,986 

.235E-X 

2.5 

.955 

.319E-1 

2.1 

.920 

-.416*^3 

1.9 

» 890 

. 705)2-4 

1.9 

.886 

-.1282-3 

1.6 

.836 

-.3392-3 

li'2 

.717 

-.6692-4 

c 9 

,576 

. 7052-4 

.5 

o367 

. 3322-4 

.5 

c 347 

.178E-4 

.3 

,217 
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TABLE 

18a. 

- THE 

MODELS FOR EXAMPLE 1 

H r 

y - 

B 0 + 

6 1 Z 4 + 

B 2 Z 3 Z 4 + E 

H 2 : 

y * 

B 0 + 

6 1 Z 4 + 

S 2 Z 3 Z 4 



+ 

B 3 Z 1 + 

S 4 Z 2 + S 5 Z 3 + E 

V 

y = 

B 0 + 

6 1 z 4 + 

B 2 Z 3 Z 4 



+ 

e 3 z l + 

V2 + B 5 Z 3 



+ 

B 6 Z 2 + 

B 7 z 3 + 6 8 Z 1 Z 4 + C 

V 

y = 

3q + 

g l z 4 + 

B 2 Z 3 Z 4 



+ 

B 3 Z 1 + 

B 4 Z 2 + B 5 z 3 



+ 

®6 Z 2. + 

B 7 Z 3 + S 8 Z 1 Z 4 



+ 

Vl + 

B 10 Z 1 Z 2 + B 11 Z 1 Z 3 + B 12 Z 2 Z 3 


+ P 1 3 z 2 z 4 + B 14 z 4 + E 





127 


TABLE 18b. - THE PRIOR MEANS FOR EXAMPLE 1 


1,0 


2,0 


3,0 


1 . 215*10 
9 . 791*10 
- 2 . 650 * 10 " 4 J 

1.064 10 
1.113 IQ " 1 
- 3.258 10” 4 
2.211 10” 3 
1.761 10” 2 
1.066 10 “ 2i 

11.76 
.1137 
- O 3376 * 10" 3 
. 3322 * Kr 2 
3114 X 10" 1 

V — 1 


c 1768 * 10 ' 
-. 1158*10 
-. 6788 * 10 " 
-. 1076*10 


-3 


~3i 


4,0 


12.70 
0.1119 
3542 * 10~ 3 
, 3194 * 10 -1 
' . 3226 > 10" 1 
. 2354 * 10”" 1 
-. 1512 * 10 “ 
-. 1277 * 10 ” 
-. 4164 * 10 “ 
. 7045 * 10 ” 
-. 3393 * 10 “ 
-. 6690 * 10 ” 
. 3323*10 


1785*10 


-4 


p- X> U> P" Ui W w 
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TAR1.E 1 8c. - THE MATRIX FROM WHICH V £ Q MAY 8E TAKEN ' 


Row 

1 

24.00000 

• 





Row 

2 

1020.000 

320700.0 





How 

3 

'23950.00 

0. 3S1425E+07 

0.2101 18E+10 




Row 

A 

300.0000 

-14125.00 

-0. 114562E+07 

108750.0 



Row 

5 

800.0000 

-127250.0 

577500.0 

72500.00 

290000.0 


Row 

6 

000. 0000 

-23950.00 

0.307475E+07 

33750.00 

135000.0 

188700.0 

Row 

7 

290000.0 

-0.380000E+07 

-0.368156E+09 

0.987500E+07 

0. 395000E+08 

0. 1S7500EH08 

Row- 

8 

183700.0 

0. 307475F.+07 

0. 5061 1 2E+08 

.0. 444750E+07 

0. 177900F+08 

0. 253050E+0S 

Row 

9 

-14123.00 

0.1 72438E+Q7 

0. 23^684 F.+09 

0. 193437E+07 

-0.229375E+07 

-0.1 14562 E+0 7 

»Kow 

10 

108750.0 

0. 193437E+07 

-0. 383297 E+09 

0. 1481 25E+03 

0.987500E+07 

0 . 4 6 S 7 50 E+0 7 

Row 

1 ). 

72500.00 

-0.229375E+07 

-0.3 83 29 7 E+09 

0.987500E+07 

0.987500E+07 

0.468750IH07 

Row 

12 

33750.0.0 

-0 . 1 14562E+07 

-0 . 35721 6E+09 

0. 468750E+07 

0.468750E+07 

0. 44 47 50 E+0 7 

Row 

13 

13300 .0 

577500.0 

-0. 336394E+09 

0. 468750E+07 

0. 187500E+08 

0. 177900E+08 

Row 

1.4 

-127250.0 

-0.301625E+07 

0. 136534E+10 

-0. 229375E+07 

'0. 380000E+07 

577500.0 

Row 

13 

0 

0 

0 

0 

0 

0 

Row 

7 

0. 775625E+10 






Row 

8 

0-343012E+10 

0. 45543 JF.+ 10 





How 

9 

-0. 823281 E+09 

-0. 35721GE+09 

0. 122473E+10 




Row 

10 

0. 193906E+10 

0.837531 E+09 

0.804 688E+07 

0. 290359E+10 



Row 

11 

0. 193906E+10 

0.857531 E+09 

-0.823281 E+09 

0. 193906E+) 0 

0.19 390 6E+10 


Row 

12 

0.911 719E+09 

0.80671 9 F.+09 

-0.383 29 7 E+09 

0.9 J.L7 1.9E+09 

0.911719E+09 

0. 85753 1E+09 

Row 

13 

0. 3646871+10 

0.322687F.+ 10 

-0.3S3297E+09 

0.911719E+09 

0. 911719E+09 

0.837531E+09 

■Row 

14 

-O.2O5016E+1O 

-0. 336394E+09 

0. 51 1906 E+09 

-0.823281E+09 

-0.823281 E+ 09 

-0. 38329 7K+09 

Row 

13 

0 

0 

0 

0 

0 

0 

Row 

13 

0. 343012E+10 






Row 

14 

-0.36S156E+09 

0. 250450E4 10 





Row 

15 

0 

0 

1.0 







